VFX 101: Greenscreen Basics, Part II

This is the third of a semi-regular series of columns I’ll be presenting on how to do some basic special effects, for people who have little to no experience. We use these effects quite frequently in our Harry Potter-based web series The Spell Tutor.

In the last column, I covered how to shoot greenscreen footage. This time I’ll be talking about how to process that footage during the editing process. I’ll try to avoid referring to specific applications since people tend to use a wide variety of programs to do special effects.

In the simplest terms, a mask is a way to let your editing program know which parts of a picture to use, and which to ignore. The basic idea is that you’ll use the greenscreen processor on your footage to generate a mask that will ignore the greenscreen and keep everything else.

So, given an example like this:

You want to end up with a mask like this:

You can think of white as the “keep” colour and black as the “ignore” colour.

Let’s work with an example that was fairly difficult to process. (You won’t learn anything from an easy example, right?) I showed you this clip sample of my pet rabbit Tyrion in the last column.
Tyrion sample

When I put the footage into my editing program and turn on the greenscreen program with the default settings, this is the mask I get:

(I should point out that I flipped the footage and moved it over because I needed Tyrion to be centered and facing the other direction - it has nothing to do with the greenscreen processing.) Obviously, this is a long way from the ideal mask we want to end up with, and the main problems are:

  1. The green doesn’t fill the entire background.
  2. The green isn’t the industry-standard greenscreen shade.
  3. The lighting is very uneven. There’s a bright spot near the floor and a dark shadow near the top. This makes everything look like different shades of green, even though the tablecloth is a uniform colour. There’s also all kinds of wrinkles and creases, which only exacerbates this problem.

The first problem is easiest to solve. I created a hand-drawn mask in my editing program to draw a rough outline that is bigger than Tyrion, but smaller than the green tablecloth. This mask will then become further refined by the next few steps we take. This is what I drew by hand:

And this is the resulting mask:

So you can see we’re already closer to our end goal by mass eliminating most of the surroundings.

On to the second problem. My greenscreen processor comes pre-programmed with the industry-standard green, but since the plastic tablecloth wasn’t that colour, I used the “sample” feature of my greenscreen processor. The idea is to click on a spot in the footage and the greenscreen processor will take the colour at that point to do the processing, rather than the pre-programmed colour. I chose a spot that was somewhere in between the brightest and darkest areas surrounding Tyrion, sort of the “average” green, if you will. This is the result, which is a little better.

Now on to the next step, where you’ll tweak the greenscreen processor’s parameters to fine-tune the results. My processor starts with the widest green range by default. You want to tweak the “low” and “high” thresholds to narrow that range and hone in on just the green parts. The more narrow you can get the range, the better, but this will be affected by the third problem I described above, the amount of variation in the background green.
I’ve found the easiest way to work is to watch the mask as you change the threshold values - either the dark areas will become pure black or the light areas will become pure white as you move the values around. The idea is that you play with both values until the mask becomes pure black-and-white, or as close as you can get it.

Now, this end result wasn’t perfect, and honestly it never will be, because I was sloppy when shooting the footage. Fortunately, I didn’t need perfection for the scene this footage was used in: Episode 2.2 “The Impossible Task”

Basically you want to take as much care as possible to avoid the three problems I listed above during the shoot. If you do so your post-production will be relatively painless if you follow the steps above. It takes practice to get it right, so experiment with what you have and good luck!


Hey, love the article! I am curious what editing software you use?

1 Like

Thank you!

I use Vegas Pro, which used to be owned by Sony but recently got acquired by Magix. I find it’s reasonably priced PC-based software. I’m actually about 5 versions behind right now, I need to look into upgrading as soon as our current season’s done.


I’m working on a project now that requires a lot of green screen and I’m going a bit buggy trying to make it look good.

1 Like

I’d be happy to help - send me a private message


Message sent! Thanks!

1 Like