Yup, that was on our national holiday on October 3rd in 2010.
We've kept paying since 1919... billions of billions... and what for? Because it was very convenient to have a single country that could be made responsible for WW1. Sure, Kaiser Wilhelm II was a militarist idiot who didn't mind going to war to get what he wanted but it's not that other European countries wouldn't have had their fair share of militarism, they all had some things so settle and went to war with flying flags and happy faces, only to find out what a war between industrialized nations really means: Total destruction without bounds and meaning.
It was the treaty of Versailles that fueled the Nazi propaganda to a great extend. That and the stab-in-the-back legend ("the German army was forced to retreat from the battlefield undefeated, it got stabbed in the back by the neo-democratic liberal politicians at home who kneeled before the Allies and their insane reparation claims and was forced to live in shame thereafter") were two major driving forces behind Hitler's rise to power.
The end of the payments also marks the end of all reparation claims that were active from 1919 to 1945 and until 2010. It not only was the Versailles treaty that got repaid, but also all foreign debt of the Reich plus a ton of claims related to the Shoa. This has been contractually secured with and by a series of official talks between Germany, the former Allies, international members of the Jewish Council/Israel and all European and Eurasian countries we went to war with in WW1 and 2. Greece had all the time in the world to bring in their claims but they didn't, and now they want us to pay another bunch of billions for something we already paid for in more than one regard.
About the economy thing, Stiglitz is a very clever man, that goes without saying, but I think that he tends to make quite scholastic arguments. I've found that to be a problem with many scientists who give their verdict based on how it should be, not how it is. In a country like Greece with its high level of corruption and lots of brotherhoods who scratch eachother's backs, a lot of political decisions influencing the economy are being made day by day that don't really make sense from the view of a dyed-in-the-wool economist but sure as hell do in the eyes of those who profit from it. Couple that with the financial crisis that hit Greece harder than other countries and you can tell that their problems cannot be solved by pumping more money in a bottomless pit without first applying serious political and social changes. Of course that is a change that has to come from within.