Let me introduce BuildMagic: https://bitbucket.org/IntelliFactory/buildmagic - get 0.0.1 via NuGet
Have you used NuGet? If not, you probably should - with aggressive backing from Microsoft, it's quickly converging to be the default package manager and binary repository for .NET, other similar projects now stand no chance (though they often have more technical merit).
Have you used NuGet package restore? You probably should - pushing binaries into source control is wicked. Especially if you are using DVCS - every fresh pull will have to get all history of your binaries. Especially if you are using Bitbucket which is quite slow on binaries.
Now, have you been disappointed by NuGet package restore requiring you to commit NuGet.exe binary to source control? If so, check out BuildMagic. It is a little workaround for the issue. Unfortunately you still have to commit something redundant (a targets file). But at least now you can say a definite NO to binaries.