Normally if you install that way (with no PREFIX) you'll install into perl's site directory, which is already in @INC.
If you can provide a little more detail, it would be helpful:
* how did you get it to install in another directory? did you use the PREFIX parameter?
* do you know where Imager/Color.pm ended up after? find or locate might be useful to find out.
* which platform are you on and which version of perl?