Regardless, deleting pre-compiled libraries out of your repository is possible and useful. The page that I got most of your information is this one:

http://stackoverflow.com/questions/1216733/remove-a-directory-permanently-from-git

git filter-branch is the command you need to use to rewrite the history. git rebase should also be able to do it, but I had no luck using it. So, most of the steps come from here:

http://progit.org/book/ch9-7.html

The steps for directories on Windows are a bit different though and I will describe them here.

1. We have a our clean repo with one file in it, run git gc and see how much space it’s taking up:

   C:\git\test>git gc
   Counting objects: 8, done.
   Delta compression using up to 2 threads.
   Compressing objects: 100% (3/3), done.
   Writing objects: 100% (8/8), done.
   Total 8 (delta 0), reused 0 (delta 0)
   
   C:\git\test>git count-objects -v
   count: 0
   size: 0
   in-pack: 8
   packs: 1
   size-pack: 4
   prune-packable: 0
   garbage: 0
   

   size-pack is the one we are interested in and it is 4K. Good.
2. Now we add boost library, lets see what happened to our size:

   C:\git\test>git gc
   Counting objects: 562, done.
   Delta compression using up to 2 threads.
   Compressing objects: 100% (553/553), done.
   Writing objects: 100% (562/562), done.
   Total 562 (delta 102), reused 8 (delta 0)
   
   C:\git\test>git count-objects -v
   count: 0
   size: 0
   in-pack: 562
   packs: 1
   size-pack: 738
   prune-packable: 0
   garbage: 0
   

   Packed size is up to 738K. This is what gets transmitted over the wire. Lets now permanently delete it.
3. First, garbage collect the repo. This is required to pack all your objects. This is absolutely necessary if you want the repo size to decrease. DO NOT OMIT THIS STEP!!!

   C:\git\test>git gc
   Counting objects: 562, done.
   Delta compression using up to 2 threads.
   Compressing objects: 100% (553/553), done.
   Writing objects: 100% (562/562), done.
   Total 562 (delta 102), reused 8 (delta 0)
   

4. Now, lets run git filter-branch. This step is taken out of Pro Git book. However there are a few caveats. First, the slashes in the path must be forward slashes (unix style), even if working out of windows command line. Second, you must specify -r argument when calling git rm.

   c:\git\test\git filter-branch --index-filter "git rm -r --cached --ignore-unmatch libs/BOOST_1_28_0"  HEAD
   
   ...
   ...
   ...
   rm 'libs/BOOST_1_28_0/type_traits/function_traits.hpp'
   rm 'libs/BOOST_1_28_0/type_traits/fwd.hpp'
   rm 'libs/BOOST_1_28_0/type_traits/ice.hpp'
   rm 'libs/BOOST_1_28_0/type_traits/object_traits.hpp'
   rm 'libs/BOOST_1_28_0/type_traits/same_traits.hpp'
   rm 'libs/BOOST_1_28_0/type_traits/transform_traits.hpp'
   rm 'libs/BOOST_1_28_0/type_traits/transform_traits_spec.hpp'
   rm 'libs/BOOST_1_28_0/type_traits/type_traits_test.hpp'
   rm 'libs/BOOST_1_28_0/utility.hpp'
   rm 'libs/BOOST_1_28_0/utility/base_from_member.hpp'
   rm 'libs/BOOST_1_28_0/utility_fwd.hpp'
   rm 'libs/BOOST_1_28_0/version.hpp'
   rm 'libs/BOOST_1_28_0/weak_ptr.hpp'
   
   Ref 'refs/heads/master' was rewritten

   Note.You should see each individual file being rewritten, if you see nothing at all, something is not right.
5. Still following the Pro Git book, we delete .git/refs/original and .git/logs. Prune and count objects:

   C:\git\test>git gc --prune=now
   Counting objects: 9, done.
   Delta compression using up to 2 threads.
   Compressing objects: 100% (4/4), done.
   Writing objects: 100% (9/9), done.
   Total 9 (delta 1), reused 8 (delta 0)

   C:\git\test>git count-objects -v
   count: 0
   size: 0
   in-pack: 9
   packs: 1
   size-pack: 4
   prune-packable: 0
   garbage: 0
   

   Yay!!! the repository is back to it’s original size.
6. Sometimes the steps above do not work and your packed size remains the same. Do not fret, there is a solution and it can be found on this page: http://stubbisms.wordpress.com/2009/07/10/git-script-to-show-largest-pack-objects-and-trim-your-waist-line/ Create an empty repository and clone the half cleaned one like so:

   C:\git\test>mkdir ..\test_clean
   
   C:\git\test>cd ..\test_clean
   
   C:\git\test_clean>git init
   Initialized empty Git repository in C:/git/test_clean/.git/
   
   C:\git\test_clean>git pull file:///c/git/test
   remote: Counting objects: 17, done.
   remote: Compressing objects: 100% (6/6), done.
   remote: Total 17 (delta 3), reused 15 (delta 2)
   Unpacking objects: 100% (17/17), done.
   From file:///c/git/test
    * branch            HEAD       -> FETCH_HEAD
   
   C:\git\test_clean>git count-objects -v
   count: 17
   size: 11
   in-pack: 0
   packs: 0
   size-pack: 0
   prune-packable: 0
   garbage: 0
   
   C:\git\test_clean>git gc
   Counting objects: 17, done.
   Delta compression using up to 2 threads.
   Compressing objects: 100% (8/8), done.
   Writing objects: 100% (17/17), done.
   Total 17 (delta 3), reused 0 (delta 0)
   
   C:\git\test_clean>git count-objects -v
   count: 0
   size: 0
   in-pack: 17
   packs: 1
   size-pack: 5
   prune-packable: 0
   garbage: 0
   

   Voila, The repo size is back down again!!! The fact that it’s 5K is probably due to the fact that we added some stuff after we added boost.