Researchers from the United States and Tanzania called for the rethink after discovering that the supplements can cause severe illness and death if they are given to children in areas with high rates of malaria.
"Current guidelines for universal supplementation with iron and folic acid should be revised," Professor Robert Black, of Johns Hopkins University in Baltimore, Maryland, said in a report in The Lancet, a medical journal.
He and his colleagues said the supplements could help iron-deficient and anaemic children but should not be given to those who aren't.
International guidelines recommend iron and folic acid supplements for children under 2 years old living in areas with high rates of anaemia.
The scientists called for the change after studying more than 24,000 young children in Pemba, Zambia. The children were randomly assigned to receive either iron and folic acid, iron, folic acid and zinc, or a placebo.
The scientists stopped the iron and folic acid arms of the study early after discovering that children taking the supplements had a 12 percent higher risk of being hospitalised with severe illness than those in the placebo group.
Iron deficiency anaemia is the most common nutritional problem in the world. It affects 4-5 billion people, according to the World Health Organisation (WHO).
Anaemia can impair physical and mental development. Most sufferers live in the developing world where the condition is aggravated by worm infections and malaria.
"Our findings indicate a potential risk of routine supplementation with iron and folic acid in pre-school children in a population with high rates of malaria and other infections," Black said.