This note in the Installing documentation may help:

"(Note: The OS X installations above place exiftool and its "lib" directory in /usr/bin, while the standard Unix "make install" described below puts exiftool in /usr/bin and the individual libraries in /Library/Perl/#.#.#, where "#.#.#" is your Perl version. If both sets of libraries exist, /usr/bin/lib takes precedence for exiftool, but /Library/Perl/#.## is the default for any other Perl scripts.)"

So you should delete /usr/bin/lib (after confirming it contains only ExifTool files) if you want exiftool to use the modules installed with "make install".

- Phil