? 
As you say only swimmer can understand how difficult it is to swim. But then it applies for every sport. There is no lesser committment in any sports. Take an example of Haile Gebrselassie
He is from the poorest country in the world Ethiopia. NO faciliities, no training, no money nothing. Still went on to win gold in 1996 and 2000. A patient of Asthma. And a marathon runner. Gebrselassie was unable to train for 3 weeks due to inflammation of his Achilles tendon during Athens. Hence he couldnot succeed.
Gebrselassie was born as one of ten children in Asella, Arsi Province, Ethiopia. As a child growing up on a farm he used to run ten kilometres to school every morning, and the same back every evening. This led to a distinctive running posture, with his left arm crooked as if still holding his schoolbooks.
Because of Beijing's air pollution levels, Gebrselassie decided to withdraw from the marathon at the 2008 Beijing Olympics. He suffers from asthma and said that running in such conditions may be harmful and even fatal to him.
Another thing one must remember is, if michael phelps was not playing for US and playing for Nepal or for that matter any other country he would not have won 8 golds. But he would have to be happy with 5 gold. Now that would make him a lesser swimmer than mark spitz or kristin otto. So he wouldn't probably regarded as the best olympian .isn't it?
I know winning gold in swimming is difficult. But if you are good enough to win one gold then probably you are good enough to win many more in single olympic. However, it is not the case of other sports.
Another point is, in olympic 1952, there were 11 events of swimming, now it is 34. even within the 100, 200m many new events have been introuduced which is speciality of phelps. Mark Spitz in his time had to compete in less number of events .
Unlike swimming, Why there is only single gold in diving, Put the gold medals for each events reverse, forward, pike, tuck in differnt degree of diffiiculty. She is as dominating in diving as michael phelps is in swimming. There is little doubt that world champion Guo JingJing would have won gold in each and every one of those.See her points in different events. Unfortunately for a diver everything counts for just one.