Colourisation Guide

Disclaimer

This document and related texts are protected under licence and is copyright Matthew Bennett, 2007.

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales License.

This document may be used with the following conditions:

You are free:

to Share, to copy, distribute and perform or transmit the work under the following conditions:

Attribution. You must give the original author credit

Noncommercial. You may not use this work for commercial purposes.

No Derivative Works. You may not alter, transform, or build upon this work.

For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to this web page.

Any of the above conditions can be waived if you get permission from the copyright holder.

Nothing in this license impairs or restricts the author's moral rights.

Content referred to in this guide is used for educational and reference purposes only. The video material has been selected for its high standards of technical quality as this is material digitally remastered from 16mm film to videotape, stabilised and converted to video quality standard.

A definition of colourisation

A summary of the colourisation process detailed here is available at:


http://en.allexperts.com/e/c/co/colourisation.htm


Colourisation (colorization) is the process of adding colour to footage which presently contains no colour data.

Footage of this type can fall into two categories, depending on the need to colourise, and the process is radically different:

1. Colour stripping, commonly referred to as Film colorization is the process where the footage was originally recorded in colour, but then certain areas of the colour data was stripped out of the frame in post production to create the effect of a film noir, or monochrome footage with certain items in the frame retaining their colour. This has been used to best effect on films such as Sin City, Kill Bill and Schindlers List and is the common use for colourisation within the media industry at present.

2. Colour addition is the process of adding colour to footage which did not previously contain any colour data and no colour data was recorded at the time of filming. The monochrome film stock, once digitised has marked out areas which have colour added to them. Examples of this have included Eisenstein's Battleship Potemkin where the flag on the battleship was hand coloured onto the film stock in one shot.

The process of colourisation involves a series of complex area tracks which are performed by computer known as Rotoscoping. This process of following a given masked area as it adjusts its shape across the shot relies on the use of Alpha, luma or colour data within the frame to determine the shape. With colour addition, this process is more complex as pixel tracking by luminance only often leads to haphazard results. Rotoscoping using this method is therefore labour intensive by the post production staff involved.

The technique is compounded further as colour information should match across the entire film. This is often difficult to achieve without a pervasive database of colour reference data used.

The interest in colourisation has seen an interest in research roles dedicated to fashion and styles for specific periods in history as colourists attempt to be truthful to the look and feel of the original material.

The debate as to the morality is complex and presently is lead by negative views formed by bad colourisation attempts in the 80s and 90s. However, as technology has moved forward, the search for more fluid techniques and better standards continues. An 'Institute for Colourisation' has been created to search for a de facto standard in the technical process which will achieve the most effective results which the media industry can use.

A community of practice is being created on this subject, which is designed to stimulate debate into colourisation techniques and processes. The colourisation wiki - colourisation.net contains a series of related documents, links, research papers and test material to stimulate debate in this growing subject. For this community a Frappr Map is also available.

Introduction

This document is in response to requests for further information on the actual colourisation project result. The aim of the project was to identify a software solution leading to a bespoke software solution. This has now been achieved and the bespoke software package created is able to manage broadcast quality materials with a small project engagement by the colourist. The major time dependencies have been rotoscoping as a method of controlling the areas to colourise and the actual selection and usage of colours to form the overall result.

This document tracks a similar method to that used presently by Matthew Bennett, using semi-professional quality software which shows the various elements of the technique.

The software used for this guide is as follows:


To start a new project

  1. Create a work folder on your hard drive e.g. “C:\colour project”. I would recommend a shortcut to this folder to also be created, placed on your desktop for convenient access.

TIP: At this stage I recommend creating two sub-folders: One for reference materials, photos etc and the other for completed renders.

  1. Move the source AVI movie you wish to colourise into the work folder.

  1. Start After Effects 7

  1. Create a new project and save this file into the work folder you created in step 1.
  1. In the project bin, add the AVI file by right clicking in the project area and selecting:
    Import > File…

  1. In the project window the file details will be provided with the dimensions and framerate of the clip. You will need this information later.

TIP: After Effects requires the presence of Quicktime Player to show the files. If the movie will not open in Quicktime, it definitely won’t in AE. If you do not have the decoding codec for the movie, then you will see a blank screen only.

In the above example, the full movie is encoded using a version of the DivX codec which is not compatible with Quicktime, but would open in Windows Media Player and Open Source players such as VirtualDub.

To correct this I edited the movie in VirtualDub, selecting only the area I wish to colourise (in this case a very short clip) and saved this as a new file called ‘clip.avi’. As no compressor is involved, Quicktime and subsequently AE are able to use the clip. This is also an alternative to selecting the part of the movie to work on as described later in this guide.

Preparing the material for first use

  1. In the project window create a new composition. Match the composition’s size and framerate to the existing clip (although as you have just added and have selected a clip, AE will use these setting to create your new composition). Name this composition.

The concept of the composition is not new – with a multi-layered timeline the editor can merge and overlap several layers, just as you would do when A-B editing in a video edit package such as Windows Movie Maker or Adobe Premiere. Equally, this is similar to layers in Photoshop in that layers have a ‘dimension’ and can be placed on top of each other.

In the timeline you will now notice that the timeline window for your new composition is now open. We will use this area extensively throughout this guide.

  1. Drag the movie clip into the timeline area. By dragging it to the setting section (on the left, not the timeline itself) it will default to placing the clip in the exact middle of the composition and at the starting time.
  1. In the composition settings window, (Composition > Composition Settings (Ctrl + K) amend the duration of the composition to match the clip, or to be just slightly longer. By doing so the timeline available to you will be dramatically reduced making it easier for you to navigate the clip. (NB: This can also be set in step 1)

Defining a work area

Before rotoscoping you will need to select an area to work on. The colourisation technique detailed here makes use of colour presets so that calculated colour information can be recycled for later clips where the same object appears. To do this, navigate to the start of the shot you intend to work on.

  1. Select the clip in the timeline and split the layer: Edit > Split Layer (Ctrl + Shift + D)
  1. Navigate to the end of the shot and split the layer.
  1. The mid section is the area you will be working on. Name this layer with a suitably memorable name. You will need to press the source name label at the top of the column to toggle from source name (the name of the movie file) to a more memorable one.

In the example below I’ve coloured the active layer we will be colourising in gold. The result should look similar to this:

  1. Make the inactive layers shy so that we can concentrate on the layer we need to work on.
    (The reason for doing this is so that we can hide inactive layers.)

  1. Now hide all shy layers:

You will now only see the active layer. Take a copy of the active layer and lock it to prevent further editing. Then, close the eye to not show this in the preview.

TIP: Why do we take a copy of the layer prior to colourising? There are two reasons for this – this layer, referred to as the ‘original’ layer is used to ensure that the greyscale detail is used within the colourisation as resulting layers only supply colour information. Most often in colourising for the film only key areas are colourised and the rest is deliberately left in monochrome. It also is a useful ‘safety’ layer in case you need to retry a layer.

Mapping and Rotoscoping

  1. Use the play controls and the scrub arrow to navigate around the clip:

  1. Add a flag to the timeline at both the start and end points so that you can quickly jump to these points by using the number keys on the keyboard. This is done by dragging a blank marker from the right ‘pot’ to the relevant area on the timeline. Use the number given to jump to that point in the timeline.

Create a duplicate of the layer for the first area you wish to colour. Layers are used to assign colours and can contain multiple masks. Masks are used to apply colour only to the masked areas.

Go to the easiest starting point you can find in the frame (this is not always the starting frame in the clip) and map the area.

TIP #1: The order the layers appear in the timeline determines the order they are used with the layers at the top of the timeline indicating the colours which will appear topmost.

TIP #2: You don’t have to be that accurate with all your masks – where an object appears in the foreground and you are working on a mask of the shape behind the foreground one, you only need to rotoscope the foreground shape on the foreground layer. For example, if you have a light ball against a dark coloured background, only rotoscope the ball. Where you have two shapes and one overlaps the other, the object behind the foreground one is obscured and so fine detail in the rotoscope is not necessary.

  1. Use the pen tool to map the area you wish to colourise. Use the convert vertex tool hidden in the pen button dropdown to swap a point between curves and lines mode. Use the black arrow to select mask points and the cursor keys to get an exact fix.

TIP: Using the Shift + cursor keys will move the point in that direction by 10 pixels and saves time.

Your work area should now look like this:

As you can see from the above illustration, I have added a marker in the mask shape so that the computer knows that at this frame this is the shape of this particular mask.

You will also notice that I have only used a small number of mask points to create the shape – this is deliberate as we do not want to over-complicate the job. We will later apply a mask feather to the are of up to 4 pixels which is sufficient for the eye to see a soft transition of each colour.

As you will notice, we also have the Dalek fins to mask. I’ll map those on a separate layer and when happy with the result merge the layers together to form one layer with several rotoscoped masks.

For now, let’s rotoscope the mask using the file RotAE1.1.jsx, a link to which is available from the colourisation.net site.

Select the mask on the frame will start the track.

Pres ‘Apply Tracker’ to automatically create tracking points matching your mask points at this frame.

Use the analyse forward / backward controls in the tracking panel to perform the tracking. Ensure that each track point tracks the shape properly. If not you may need to hide track points and work on them individually. The hide option can be found in the layer details:

Motion trackers > Tracker (number) > Track point (number) and close the eye of any you do not want to track right now.

When you are happy with the results, select the tracker number and the mask to apply the mask. The mask will now be rotoscoped.

TIP #1: This can be a laborious process and it is not necessary to track every single frame. One way around this is to adjust the framerate of the composition to 1/5th of its normal setting and then rotoscope. Then reset the framerate to normal. By doing so you’ll have a keyframe every 5 or 6 frames rather than every one.

TIP #2: If there is little movement in the shape, or shadows get in the way of the tracker working effectively you will have to manually set mask shape keyframes by pressing the keyframe button on the timeline as below:

When you are happy with your masks and have mapped all elements which are to be the same colour at that level within the topology of the picture, then we are ready to colourise.

Applying a colour range

  1. Right-click the layer in the timeline (left side of the pane) and select Blending Mode > Color
  1. Set the mask feather to no greater than 4 pixels. This will help to blend the colours around the edges.

  1. Select the layer in the timeline and press F3 to open the effects panel. Right-click in the effects panel and select Walker Effects 2.2 > colorize

With the panel now open ensure that the composite mode (the blend) is also set to ‘color’.

TIP #1: Select a preset with a number of colours to determine which greyscale areas are affected by which colours. Use a preset such as ‘primaries’ to ‘get to know your shape’.

It is best to use the plug-in in 5-colour mode although it is possible to colourise by only selecting one. The photo-realism is created by accurate use of a multiple of colours.

If you have reference material in colour such as videotape or film footage from an area which is in colour and does not need colourising it would be advisable to take a ‘patch’ of the area you wish to colourise in your monochrome footage (e.g. actor 1’s hair) and attempt to match the original colour. This is done by correct selection of colours but also adjustment of the saturation of the colour which is done by adding more white to the colour. It is advisable to avoid the opacity % option because you then have the option to fine tune the colour patch later on.

Dalek dome using 1 colour (pale sky blue)

Dalek dome using 5 colours

Here is the completed dome section with light added. The light was tested and found to have two key colours – the top of the light is a reflection from the studio lights so has a yellow sheen, where the main light is plastic and from reference is a 50/5- mix of pale orange and pink as below:

Finalising and test rendering

By adjusting the opacities of the colour patches you will then be able to ensure that all colours then work in relation to each other.

The procedure will then be to render the result as an uncompressed AVI file (video only, uncompressed).

Please ensure that all colour patches created are saved as colour presets as these will be recycled in later shots.