The Black & White Adjustment Layer is meant to create black and white photos. However, you can use it to control the luminescence values of the colors of your image.
To do so, create a Black & White Adjustment Layer, and set the Blending Mode to Luminosity.
This change will bring the color of the image back. Then you can use the sliders in the Properties Panel to control the darkness or lightness of each color.