Learn these three amazing techniques to make better cut-outs in Photoshop composites.
Table of contents
Channel Based Masks Instead of Blending Modes
If you have a black background with a white foreground and you want to keep the white areas, changing the Blending Mode will not be the best solution.
Using a Blending Mode to hide the background will not allow you to add Layer Styles, filters, or adjustments to just the visible areas.
To create better cut-outs in Photoshop with these types of photos and gain transparency, go to the Channels panel.
For a colored image, select the channel with the starker contrast between the foreground and the background.
For a black and white image such as this one, the channels will all look the same so you can select any channel.
Press Ctrl (Windows) or Command (macOS) as you click on the selected channel’s thumbnail to load the bright pixels as a selection.
Click on the RGB channel to enable all the channels.
Go to the Layers panel, click on the New Adjustment Layer icon > Solid Color.
On the Color Picker window, set the color to white (#ffffff) and press OK to exit the window.
This will look similar to the result if you had set the Blending Mode to Screen, but the big difference is this one has a transparency that will make it easy to work with.
If you bring up the Layer Style window by double-clicking on the layer’s side, you can apply any Layer Style such as emboss, and it will only apply on the white pixels.
Remove Edge Halos or Fringing For Better Cut-Outs in Photoshop
After removing a background from a photo, you may see edge halos or fringing. These distracting oulines can completely destroy your composite.
To remove fringing or halos from your mask edges, you can try this technique.
Use the Minimum Filter to Contract the Layer Mask
First, select the Layer Mask thumbnail.
Go into Filter > Other > Minimum.
The Minimum filter allows you to contract the Layer Mask with the Radius slider.
Notice that as you adjust the radius, the mask edge halos start disappearing. Adjust your Radius accordingly.
Set the Preserve algorithm to Roundness if you’re dealing with organic subjects such as people, animals, or plants.
Set the Preserve to Squareness if you’re dealing with inanimate subjects such as cars, buildings, or products.
Then, press OK to apply the filter and exit the window.
To learn more about how this filter works, check out my crash course on the Minimum and Maximum Filter!
Adjust the Layer Mask Edges with Select and Mask
From the Layers Panel select the Layer Mask Panel, and from the Properties panel, click on the Select and Mask button.
Under Global Refinements, use the Smooth, Feather, and Contrast sliders to refine the Layer Mask’s edges.
Then, press OK to exit.
The refinement of the hair is best when it’s left as the last step.
Click on the Select and Mask button again to return to the workspace.
On the Toolbar, select the Refine Edge Brush tool and use this tool to paint around the hair are to refine its edges and reveal the individual hair strands.
When you’re done, press OK to exit the window and apply the changes.
BONUS TIP!
To deal with the pesky flyaway hairs, create a New Layer and press Ctrl Alt G (Windows) or Command Option G (macOS) to clip it to the layer below.
Select the Brush tool and hold the Alt (Windows) or Option (macOS) key as you select on the hair color to sample it and set it as the Foreground Color.
Now, use the Brush tool to paint on the flyaway hair strands.
For hair strands on the hair’s inner part, create a New Layer and press Ctrl Alt G (Windows) or Command Option G (macOS) to clip it to the layer below.
Hold the Alt (Windows) or Option (macOS) key as you select on the lighter hair color to sample it and set it as the Foreground Color.
Once again, paint over these edges.
Then, change the Blending Mode of the layer to Darken to apply the color to the brightest pixels. If it appears too strong, reduce the Opacity.
Masking Hair in Busy Backgrounds
When you are dealing with a curly-hair subject on a busy background, here’s how you can do it:
Select the Subject layer.
Under Quick Actions, on the Properties panel, click on the Remove Background button and let Adobe Sensei, an Artificial Intelligence, to do the work.
In this case, it achieves a great job despite the hair texture. If you want to see the result more clearly, create a Solid Color fill layer, and on the Color Picker window, set the color to a dark grey.
With the Subject layer selected, press Ctrl J (Windows) or Command J (macOS) to duplicate the layer.
Select the duplicated layer’s Layer Mask thumbnail.
On the Properties panel, click on the Select and Mask button.
Once on the Select and Mask workspace, set the View mode to Black & White.
Select the Refine Edge tool and paint around the edges to further refine the hair strands and remove or prevent the blurry areas. The Refine Edge Tool is a fantastic tool to make better cut-outs in Photoshop.
In this case, where the blurry areas are unpreventable, you may exit the Select and Mask workspace, delete the duplicated Layer, and start anew.
Using an Image to Create a Preset Brush for Hair
With a new duplicated Layer, use the Laso tool to create a loose selection of the hair area that is difficult to select.
Set the Background Color as black (#000000).
With the Layer Mask thumbnail selected, press Ctrl Backspace (Windows) or Command Delete (macOS) to fill the selected area with black and hide that part of the image.
Now, you can start the process of taking an image of a hair that can be easily-selected and creating a brush out of it.
Open the image on a new tab.
Use the Crop tool to crop the areas you don’t need.
Go to the Channels panel and find out which channel has more contrast between the foreground and the background. In this case, it’s the Blue channel.
Duplicate the Blue channel by clicking-and-dragging it towards the New Channel icon at the bottom of the panel.
Select the Blue channel copy and press Ctrl I (Windows) or Command I (macOS) to invert the color.
From here, the areas you want to create as a brush should be in white, and everything else should be in black.
Choose Image > Adjustments > Levels.
Adjust the input levels to make the background black and everything else to be in white.
Use the Lasso tool to create a loose selection of the remaining grey areas.
With the Foreground Color set to white (#ffffff), press Alt Backspace (Windows) or Option Delete (macOS) to fill the selection.
Load the bright pixels as a selection by holding Ctrl (Windows) or Command (macOS) as you click on the Blue channel copy thumbnail.
Click on the RGB channel to return to the original colors.
Go to the Layers panel and disable the image layer.
Create a New Layer.
Set the Foreground Color to black (#000000).
Press Alt Backspace (Windows) or Option Delete (macOS) to fill the selection with the Foreground Color.
Then, press Ctrl D (Windows) or Command D (macOS) to deselect the selection.
Create a New Layer and fill it with the color white. Since white is set as the Background Color, you can also press Ctrl Backspace (Windows) or Command Delete (macOS).
Finally, you can turn this into a brush by choosing Edit > Define Brush Preset.
Then, set an appropriate name for the new brush and press OK to save.
Return to the previous working document, and you can start using the Brush to “paint” in the hair’s edges.
For a more realistic look, create a New Layer underneath the current layer.
Select the hair’s lighter color and use the color to “paint” in the hair’s underlying texture.
Download The Hair Brush!
To make better cut-outs in Photoshop on photos with busy backgrounds you can click the link below to download the PTC Hair Brush.
319 – PTC Hair Brush (2471 downloads)Related Tutorials
If you enjoyed this tutorial, make sure to follow PTC on YouTube! And if you create something using this tutorial, then share it on social media with the hashtag #PTCvids for a chance to get featured!
Final Image
Drag The Slider To See Before & After
