Africa could feed itself if trade restrictions were reduced and fertile land was put to good use, according to the World Bank.
Just 5% of African cereal imports come from other African countries, it said.
Removing cross-border barriers would free up trade, reduce prices and generate billions of dollars for African governments, it added.
The bank estimates there are almost 20 million people affected by hunger in the Sahel region of West Africa.
"Too often borders get in the way of getting food to homes and communities which are struggling with too little to eat," said Makhtar Diop, World Bank vice-president for Africa.
Trade restrictions also raise the price of food to consumers and reduce the incentive for farmers to produce, as they receive only a small proportion of the end price.
High transport costs, including roadblocks and bribes at border posts, as well as strict rules restricting the use of high yielding seeds and more effective fertilisers, exacerbated the problem, the bank said.
"The challenge is how to create a competitive environment in which governments embrace credible and stable policies that encourage private investors and businesses to boost food production across the region," said the bank's Paul Brenton.
Food prices are rising across the world following severe droughts in many countries and are approaching record levels seen last year and in 2008, a year which saw food price riots in many African nations.
The bank does not believe, however, that the situation is as dire as in previous years, because inventories of crops are higher and major producers have not introduced export bans.