As a child, Sheikh Yamani received both a traditional general schooling and an education in Islamic Law from eminent jurists in the Holy Mosque. He then entered the King Fouad I University in Cairo, graduating in 1951.

His working life in the Kingdom commenced with a short-term position as a government official, but this was curtailed to take up graduate studies at New York University where, in 1955, he was awarded a Master's Degree. He received a Master of Law from Harvard University in June 1956. Upon his return, he took up the post of Legal Adviser to both the tax and the Oil and Minerals departments.

In 1957, he inaugurated Saudi Arabia's first law office and following this he became Legal Adviser to the Council of Ministers under Crown Prince Faisal. In 1959, he was appointed a Minister of State and a member of the Cabinet. However, he continued to teach Law part-time at the King Saud University in Riyadh (1957-60).

Sheikh Yamani was named Minister of Petroleum and Mineral Resources in March 1962, an office he held until October 1986. In 1963, he established the University of Petroleum and Minerals in Dharan and remained its Chairman until 1976. More recently, Sheikh Yamani founded the Centre for Global Energy Studies in London, which was officially launched in January 1990 to carry out objective studies dealing with the global energy question.

On a more cultural note, he has set up a foundation for the preservation and publication of old Arabic and Islamic manuscripts.

Sheikh Ahmed Zaki Yamani has several honorary doctorate degrees from universities in North and South America, Europe, Asia and Africa. He has also been awarded a selection of meritorious decorations from many countries.