Not necessarily.
Breast size also has some relationship mainly with genetics. If the mother has small breasts, the child is likely to have small breasts. Also, if an adolescent girl is malnourished, her breasts can become smaller. If a woman loses too much weight, this can also lead to small breasts.