I came across a post in the MSDN Forums on how to delete a project from your TF Service.
“For some time now Buck Hodges has had a nice BLOG post about deleting team projects from Team Foundation Service collections. Not trying to reinvent the wheel but I thought it valuable to have it here in our support forum, too.
Basically, to delete a team project from your Team Foundation Service collection you will need Visual Studio 2012 (or just Team Explorer 2012, which is free here). These provide you with the "tfsdeleteproject.exe" command line tool, which you run thusly:
tfsdeleteproject /collection:https://<Your Collection>.VisualStudio.com/DefaultCollection <"Project Name">
Don’t forget the "/DefaultCollection" bit.”
Thanks Trevor for providing this.