In this Photoshop tutorial, you will learn how to cut out trees in Photoshop using three powerful techniques!
This information-packed Photoshop tutorial will be filled with useful tips and tricks that you will be able to apply to any masking job, not just trees!
You’ll learn about the “Blend If” command, and how you can create true transparency with it.
You’ll also learn about the Defringe command which allows you to remove edge fringes and halos, and you will learn about the tree generator and how you can use it to create custom brushes to give your trees a virtual trim.
Blend If & Channel Explanation
A tree already isolated from its background and other elements is an excellent image to start. Please make sure the layer is unlocked and double-click on it to bring out the Layer Style window.
Use the Blend If sliders to hide or reveal pixels based on the luminosity of each RGB channel.
You may also go to the Channels panel and see which channel has a sharper contrast between the foreground and background and adjust it in the Blend If sliders.
Use the Blue Channel to Cut Out the Tree
In this example, the Blue channel has a sharper contrast, so use the Blend If sliders to hide the brightest pixels in that channel.
Click-and-drag on the white point, and as you drag it inwards, it should hide the bright pixels which mostly belong to the sky to cut out the tree against its background.
Once that is set, press OK to save the cutout.
PRO TIP: Hold Alt (Windows) or Option (macOS) as you click on a point to split it and create a smoother transition.
Create True Transparency From the Blend If Blend
From the layer thumbnail in the Layers panel, you can see that there is no real transparency, it is just a blend.
You can convert the layer into actual transparency by converting it into a Smart Object. Right-click on the layer > Convert to Smart Object.
Immediately, you’ll notice the checkerboard pattern that illustrates transparency in the layer thumbnail.
Use the Lasso Tool to Make a Loose Selection
Used in this 2nd method on how to cut out trees is an image of a tree against a sunset with a foggy scenery. What makes this distinct from the first photo is the presence of multiple highlights surrounding the trees, so the Blend If method won’t work best with these kinds of images.
Start by creating a channel-based mask.
Go to the Channels panel and determine which channel has a sharper contrast between the foreground and the background. In this case, it’s the Blue channel.
Duplicate the Blue channel and use the Lasso tool to create a loose selection around the tree.
Invert the Channel
Right-click > Select Inverse to invert the selection.
With white(#ffffff) as the Foreground color, press Alt Backspace (Windows) or Option Delete (macOS) to fill the selection with the color.
Then, press Ctrl D (Windows) or Command D (macOS) to deselect.
Use the Dodge and Burn tool to Isolate the Tree in the New Channel
To accurately cut out trees in Photoshop, select the Dodge and Burn tool to brighten and darken existing pixels in the image.
Since the golden rule of masking is, “white reveals, black conceals,” invert the colors to turn the tree into white by pressing Ctrl I (Windows) or Command I (macOS).
Then, start using the Burn tool with the Range set to Shadows. Click-and-drag around the dark pixels to make them darker and further isolate the tree.
On the Options bar, the higher the rate of Exposure, the stronger the effect will be. The lower the rate of Exposure, the more control you have to build upon the Shadows.
You can use the Brush tool and paint with black(#000000) to mask them for stubborn areas.
Reversely, use the Dodge tool to accentuate the tree further. Set the Range to Highlights and adjust the exposure, then start painting along the tree. You can also use the Brush tool with white(#ffffff) set as the Foreground color to reveal it in the mask.
Load the Selection
Back on the Channels panel, press Ctrl (Windows) or Command (macOS) as you click on the Blue copy channel thumbnail to load the tree as a selection.
Click on RGB to retrieve the original colors and go back to the Layers panel.
Then, click on the New Layer Mask icon to hide the background except for the tree.
Create a Solid Color fill layer and set the color to a light grey. Then, place the Solid Color fill layer right beneath the masked tree.
Use the Levels Adjustment to Refine the Mask
If there appears to be halo or fringing along the edges of the tree, select the Layer Mask thumbnail then choose Image > Adjustments > Levels.
Click-and-drag the dark point inwards until you can no longer see the fringing along with the leaves.
Use the Minimum Filter to Contract the Mask and Remove Halos
Another method of achieving the same effect is to choose Filter > Other > Minimum to contract the mask by adjusting the Radius value of the pixels. If you want to learn more about how to use the Minimum and Maximum filter for your masking, click here!
Use the Defringe Command to Remove Fringing From the Mask
If the fringing is stubborn and wasn’t removed by the Minimum filter, duplicate the layer > select the Layer Mask thumbnail of the duplicated layer > Apply Layer Mask.
Choose Layer > Matting > Defringe.
The Defringe command looks at the edge pixels and swaps it with the color of the surrounding pixels, so set the appropriate Width value of the pixels and press OK.
Select the Tree layer and set the Blending Mode to Darken.
Then, clip the layer to the layer directly below it by pressing Ctrl Alt G (Windows) or Command Option G (macOS) to achieve a better effect.
Use the LassoLaso Tool to Make a Rough Mask
Unlike the technique used earlier, determining which channel has a sharper contrast between the foreground and the background will not work for this specific image since the surrounding environment is also filled with trees.
Start by selecting the layer and use the Quick Selection tool to create a tree trunk selection.
Then, select the Lasso tool and hold the Shift key as you click-and-drag around the tree to add it to the current selection.
But for this one, draw inside of the tree and leave a wide gap between the draw selection and the edges of the leaves.
With the selection active, click on the New Layer Mask icon to create a Layer Mask.
Use the Tree Generator to Create a Tree
One of the fun parts of cut out trees is being able to use the Tree Generator as a reference.
Choose File > New and create a new blank document.
On the new document and choose to Filter > Render > Tree.
On the Tree window is where you can find different types of trees under the Base Tree Types’ drop-down menu, and select the appropriate tree that matches the one in the image or looks closest to it.
Turn Your Tree Into a Brush
If you want to learn how to cut out trees in Photoshop, you might also want to learn how to turn them into a brush.
Use the Lasso tool to create a loose selection of the trunk area then press Backspace (Windows) or Delete (macOS) to delete those pixels. You can also redo this step for areas that you don’t like.
Next, duplicate the layer by pressing Ctrl J (Windows) or Command J (macOS) and transform the duplicated layer until it looks like a ball of leaves without the trunk.
With the duplicated layer selected, press Ctrl E (Windows) or Command E (macOS) to merge it with the original layer.
Choose Image > Adjustment > Levels and use the points to transform the layer’s colors into a solid black, this silhouette will be the key to making it into a brush.
Choose Edit > Define Brush Preset. Please give it a name, and now, you finally have your tree brush.
Use the Brush Settings to Customize Your Brush
Feel free to try out your cut out tree brush by painting around the empty canvas. For a more randomized texture or stroke, open the Brush Settings and enable Shape Dynamics.
Adjust the Angle Jitter and Size Jitter sliders to achieve a random stroke without any repetitions.
Paint on Your Mask with Your Custom Tree Brush
Go back to the original working document and activate the Brush tool.
Since you are using a Layer Mask, set the Foreground color to white(#ffffff) and start painting with your new tree brush to reveal the tree’s edges with a more realistic appearance.
PRO TIP: Use the [ or ] key to increase and decrease the Brush size for a more natural look.
If the edges of the tree start to reveal a new original background, create a new layer, and clip it to the layer below it to have a more precise result of cut out trees in Photoshop.
Then, use the Clone Stamp tool and press Alt (Windows) or Option (macOS) to set the sample source and paint along those unwanted edges to match the pixels inside the tree.
Those are the 3 different ways you can cut out trees in Photoshop that you can use efficiently for your next photo composite!