Apply a threshold to the transparency of a smart object layer


I have a smart object layer whose content is partly transparent. How can I create an effect such that pixels with alpha greater than some value become opaque, and other pixels become fully transparent?

I know that I could stack up 15-20 copies of the SO to get an opaque version, but I can't figure any useful way to mask the result.

Note: I'm looking for a way to set up a PSD such that the results of this threshold effect get applied dynamically, without manual steps. I know that it's possible to manually extract a layer's transparency, edit it, and then reapply it as a mask, but I'm looking for a setup where I won't have to re-execute manual steps every time the content of the smart object changes.

Example image:

Sample image


Long story short, what I was going for here isn't possible. Photoshop does let you manipulate a layer's transparency, but only if you manually extract it into a color layer and later re-apply the colors as a mask. It's not possible to get similar results with layer adjustments and built-in filters, so there's no way to get the effect without executing manual steps or saved actions every time the source layer changes.

I'll go ahead and mark @AAGD's answer as correct since it's the best manual way of getting the effect I hoped for.

2/25/2016 6:05:00 AM

Accepted Answer

I'd say having the alpha as a channel would help. This could then be manipulated with controls like Threshold or Levels. To get the alpha cmd-click (on mac) the layer icon of your smart object. This will create a selection. Then create a new channel. The selection will be translated into a grayscale representation of your transparency. Edit the channel, make it a selection again and with this selection you can create a mask on your non-transparent (20 copies merged) layer.

2/15/2016 6:56:00 PM

I can see one problem right off the back with what you are trying to do. You can't change the internal mask of a Smart Object without editing the smart object itself. So you will ALWAYS have transparency inside the smart object to contend with when editing outside the Smart Object. This should hopefully become more clear by the end of this answer.

With that said, there is a way to get close to what you want to do without editing the Smart Object itself, that can be scripted into an action, and run as a batch process or as a droplet. Below, I will walk through the steps to create a Photoshop Action to do this

For this example we will use the purple-green-orange gradient as our RGB image, and this black to white gradient as our mask with various levels of transparency.
Purple - Geen - Orange imagemask for PGO image

For this example these images results in a Smart Object that looks like this in Photoshop.

Given this as an example here's what I suggest:

  1. Select the Smart Object layer, and chose menu Select -> Load Selection...
  2. In the Load Selection Dialog make sure the channel is set to Layer 0 Transparency. This should actually be the only thing you an choose if the only item in the current Photoshop file is the Smart Object itself.
    For my example this results in a "marching ants" selection like this.
  3. With the Smart Objects compound transparency selected choose menu Layer -> Layer Mask -> Reveal Selection.
    This will create a Layer Mask for the Smart Object that matches the Smart Objects compound transparency. This will have the effect of making the image look MORE transparent as you are now adding a transparency to a transparency.
  4. With the new Layer Mask selected choose menu Image -> Adjustments -> Levels... to bring up the Levels dialog.
  5. In the Levels dialog you want to set your Black Input and White Input points to be together. You can NOT set the Black and White points to the same number. They MUST be separated by a least a single value that represents the Grey Point.
    For my example, I want every level of transparency below 50% grey in the mask to be completed transparent, and any value above 50% grey to be opaque. This means I set the Black Input to 127 and the White Input to 129. With 128 (or 50% grey on a 256 bit scale) between them.
    This results in a mask that is now clipped to hard values of transparent or opaque, BUT because we haven't modified the actually Smart Objets transparency itself the image still has transparency in the opaque side.
    This is where we hit the snag in NOT editing the Smart Objects transparency directly. You need a background color for the transparency in the Smart Object to mix against so that it's not transparent any more. You basically need to choose a white or black backing image that it masks onto. But you still need to use the new transparency you just created.
  6. With the layer mask still selected, repeat Step 1 and 2 and do a menu Select -> Load Selection..., but make sure you have chosen the Layer 0 Mask.
  7. With this new selection go to menu Layer -> Group Layers
    This results in the Smart Object with its layer mask being added into a group folder.
  8. You now need to generate a mask layer for the group folder. The selection you made in Step 6 should still be active so you can create the groups layer mask. This is the same as you did in Step 3.
    This should result in your Smart Object inside a group folder with a mask of it's own.
  9. Now we need to add in the solid White or Black backing that will act as the backing for the opaque area for the parts of the Smart Object that are still showing transparency. White the group folder selected go to the menu Layer -> New Fill Layer -> Solid Color...
  10. The New Layer dialog will appear, hit OK.
  11. Choose White (you can choose black if you want) from the color picker.
    This generates the new Solid Color layer. But it's most likely above your Smart Object.
  12. Move the Solid Color layer below the Smart Object while keeping it in the group folder.
    This results in a Smart Object that is now masked to a specific transparency range that is either transparent or opaque, and that doesn't change the original Smart Object itself. 20

You are now done. If you recorded this as an action you can now repeat this with any Smart Object.

But here's the big rub, your Smart Object is being modified by the backing image that is white or black resulting in distinct opaque and transparent areas. If you were to do the levels call that modifies the mask within the smart object you would get a different image because you aren't using a backing image, rather you a modifying the original mask itself. Here's a comparison of the process I just outlines above vs editing the mask within the Smart Object itself. Notice that the color in the Smart Object masking is begin muted by the white background image, vs going into the Smart Object itself and modifying the mask itself.
21 VS. 22

There might be anther way to do this that I don't know about, but this is the best solution I've been able to come up with to solve your quandary.

Here's the process I outlined above run with a similar image to your example image.

Here's a link to download my Photoshop Smart Object Mask Mod Action if you want.

This procedure will work for a classic Photoshop automation workflow, but probably won't work when using the newer Photoshop CC Generator feature.