Reaper Api - Video Processor Plugin(Reascripts)

hacked out, written and compiled by Meo Mespotine(mespotine.de) for the Ultraschall.FM-Project.

Index

Introduction
Coding Introduction
special variables
project_time: project time in seconds
project_tempo: current tempo in BPM
project_ts_num: current time signature numerator
project_ts_denom: current time signature denominator
project_time_qn: current project position in QN
time: item time in seconds (if in item)
framerate: project FPS (30.0, 29.97, etc)
project_w: project preferred video width
project_h: project preferred video height
project_wh_valid: set nonzero if project_w/project_h reflect actual project setting (otherwise could be media-defined)
colorspace: current rendering colorspace, e.g. 'RGBA', 'YV12', or 'YUY2'. you can override this with extreme caution.
param_wet: if in FX form, wet/dry mix of effect.
param1..param16: parameters, you can name these by adding lines: //@param [:varname] 'name' [defval minval maxval centval step].
gfx_r: current drawing color (red 0..1)
gfx_g: current drawing color (green 0..1)
gfx_b: current drawing color (blue 0..1)
gfx_a: current drawing alpha (0..1)
gfx_mode: 0=normal, 1= additive, 3=mul (works different in YUV vs RGB)
gfx_dest: destination image handle, or -1 for main framebuffer
functions
input_count() returns number of inputs available (total), range [0..n)
input_track_count() returns the number of available inputs on discrete tracks
input_track(x) returns input for bottommost item or FX on discrete-track x
input_track_exact_count() returns the number of tracks above the current track that could possibly contain video items.
input_track_exact(x) returns input for bottommost item or FX on track relative to current track.
input_next_item(x) returns the next input after x which is on a different item or track
input_next_track(x) returns the next input after x which is on a different track
input_ismaster() returns 1.0 if current FX is on master chain, 2.0 if on monitoring FX chain
input_info(input, w, h[,srctime, wet, parm1, ...]) returns 1 if input is available, sets w/h to dimensions.
gfx_img_alloc([w,h,clear]) returns an image index for drawing .
gfx_img_resize(handle,w,h[,clear]) sets an image size.
gfx_img_hold(handle) (cheaply) retains a read-only copy of an image in handle.
gfx_img_getptr(handle) gets a unique identifier for an image, valid for while the image is retained.
gfx_img_free(handle) frees an earlier allocated image index.
gfx_img_info(handle,w,h) gets dimensions of image.
gfx_set(r,g,b[,a=1,mode=0,dest]) - updates r/g/b/a/mode always.
gfx_blit(input[,preserve_aspect=0,x,y,w,h,srcx,srcy,srcw,srch]) copies input to framebuffer.
gfx_fillrect(x,y,w,h) fills a rectangle
gfx_procrect(x,y,w,h,channel_tab) processes a rectangle with 768-entry channel table [256 items of 0..1 per channel]
gfx_gradrect(x,y,w,h, r,g,b,a [,drdx,dgdx,dbdx,dadx, drdy,dgdy,dbdy,dady]);
gfx_rotoblit(srcidx, angle [,x, y, w, h, srcx, srcy, w, h, cliptosrcrect = 0, centxoffs=0, centyoffs=0])
gfx_deltablit(srcidx, x,y,w,h srcx,srcy, dsdx, dtdx, dsdy, dtdy, dsdxdy, dtdxdy[, dadx, dady, dadxdy])
gfx_xformblit(srcidx, x,y,w,h, wdiv, hdiv, tab[, wantalpha=0]) - tab is wdiv*hdiv*2 table of src points.
gfx_keyedblit(input[,x,y,w,h,srcx,srcy,kv1,kv2,kv3,kv4]) chroma-key blits, using the source color as key.
gfx_destkeyedblit(input[,x,y,w,h,srcx,srcy,kv1,kv2,kv3,kv4]) chroma-key blits, using destination color as key. ignores gfx_a and gfx_mode.
gfx_setfont(pxsize[,#fontname, flags) - sets a font.
gfx_str_measure(#string[,w,h]) - measures the size of #string, returns width
gfx_str_draw(#string[,x,y,fxc_r,fxc_g,fxc_b]) - draw string
rgb2yuv(r,g,b) -- utility, converts r,g,b to YUV, does not clamp [0..1]
yuv2rgb(r,g,b) -- utility, converts YUV to r,g,b, not clamping [0..1]

Introduction

Since Reaper 5, Reaper has proper video-editing support. In addition to that, Reaper has a plugin, called "Video Processor Plugin", that is focused on video-effect handling.

Though it is still in it's very early stages, it supports already a handful of useful effects that can be selected via the presets-selection.
It is also programmable. It opens up a small IDE-window, as part of the "Video-Processor-Plugin"-UI, in which you can put Eel-code(no Lua or Python!), that processes the video.

Unlike with other scripts, the scripts for the video processor plugin do not support the standard-reaper-API-commands, but have an own set of variables and functions. The list of functions can be reached by clicking into the IDE-window of the "video processor plugin" and hitting the F1-key.

Some general notes on the video-processor-plugin:
It has a very limited set of programmable UI-elements, means, only a knob (refer the param-variable for more details) at the time.
The plugin has also sometimes the behaviour of "added effects". That means: If you have several tracks whith video items and you put a video processor plugin on every track as track-FX, they do not only influence the video-items in this track. In fact, they also influence the video-items on the other tracks as well. If you have 10 tracks and every track has a video-effect as track-FX that turns the brightness down by 10%, the resulting image will be a black one. All effects added up each other (10 times -10% brightness).
This effect can be circumvented by putting the video-processor-plugin as item-FX, NOT as track-FX.

Coding Introduction

To write code, load the Video Processor Plugin into item-FX of the video or track-FX of the track containing your video. It opens a editor window, in which you can write the code. Hit F1 for commands and variables valid for use in the video processor plugin.
Next to it, on the left side of the editor-field, your parameter-knobs will appear(if you've defined some).
Code for the Video-Processor-Plugin must be written in EEL, not in Lua or Python.

Example code, with a knob for turning the video on and off:


                // A small demo-script that turns video on&off
                //@amp;param1:VideoOnOff 'Video Off(0) On(1)' 1 0 1 0.5 1

                frame=input_track(0); // variable frame set with the videoframe from the first 
                                      // videoitem in the project at a given position
          
                gfx_fillrect(0,0,project_w,project_h); // deletes last frame, by putting a black rectangle on top of it
                                                       // to prevent from having the last frame(s) be shown indefinately

                //Now, put the picture to the framebuffer -> outputting the video
                gfx_blit(frame,  // the current video-frame
                            1,   // don't preserve aspect-ratio
                            0,0, // put upper right corner of the video at position 0,0
                            project_w*VideoOnOff,project_h*VideoOnOff // size of the video width, height, 
                                                                      //   multiplied by the VideoOnOff-Parameter
                                                                      //   1=video squeezed to 100% height & width 
                                                                      //   0=video squeezed to 0% height & width
                            );

Special Variables


project_time

project time in seconds


project_tempo

current tempo in BPM


project_ts_num

current time signature numerator


project_ts_denom

current time signature denominator


project_time_qn

current project position in QN


time

item time in seconds (if in item)


framerate

project FPS (30.0, 29.97, etc)


project_w

project preferred video width


project_h

project preferred video height


project_wh_valid

set nonzero if project_w/project_h reflect actual project setting (otherwise could be media-defined)


colorspace

current rendering colorspace, e.g. 'RGBA', 'YV12', or 'YUY2'. you can override this with extreme caution.


param_wet

if in FX form, wet/dry mix of effect.


param1..param16

parameters, you can name these by adding lines //@param [:varname] 'name' [defval minval maxval centval step].


gfx_r

current drawing color (red 0..1)


gfx_g

current drawing color (green 0..1)


gfx_b

current drawing color (blue 0..1)


gfx_a

current drawing alpha (0..1)


gfx_mode

0=normal, 1= additive, 3=mul (works different in YUV vs RGB)
add 0x100 for blit() to enable filtering (if possible)
add 0x10000 to use source alpha (only valid when using RGBA colorspace)
add 0x40000 to use extra clamping (for out of range alpha/gradient values)
add 0x80000 to treat gfx_r/gfx_g/gfx_b as YUV values (in non-RGB colorspace)


gfx_dest

destination image handle, or -1 for main framebuffer


Functions


input_count()

returns number of inputs available (total), range [0..n)


input_track_count()

returns the number of available inputs on discrete tracks


input_track(x)

returns input for bottommost item or FX on discrete-track x (0 is first track with video item above current, etc)


input_track_exact_count()

returns the number of tracks above the current track that could possibly contain video items.


input_track_exact(x)

returns input for bottommost item or FX on track relative to current track. Returns -1000 if track does not contain any video items at the current time, or -10000 if no further tracks contain video.


input_next_item(x)

returns the next input after x which is on a different item or track


input_next_track(x)

returns the next input after x which is on a different track


input_ismaster()

returns 1.0 if current FX is on master chain, 2.0 if on monitoring FX chain


input_info(input, w, h[,srctime, wet, parm1, ...])

returns 1 if input is available, sets w/h to dimensions. If srctime specified, it will be set with the source-local time of the underlying media. if input is a video processor in effect form, automated parameters can be queried via wet/parm1/etc.


gfx_img_alloc([w,h,clear])

returns an image index for drawing (can create up to 32 images). contents of image undefined unless clear set.


gfx_img_resize(handle,w,h[,clear])

sets an image size (handle can be -1 for main framebuffer). contents of image undefined after resize, unless clear set. clear=-1 will only clear if resize occurred.


gfx_img_hold(handle)

(cheaply) retains a read-only copy of an image in handle. This copy should be released using gfx_img_free() when finished. Up to 32 images can be held.


gfx_img_getptr(handle)

gets a unique identifier for an image, valid for while the image is retained. can be used (along with gfx_img_hold) to detect when frames change in a low frame rate video


gfx_img_free(handle)

frees an earlier allocated image index.


gfx_img_info(handle,w,h)

gets dimensions of image, returns 1 if valid (resize if inexplicably invalidated)


gfx_set(r,g,b[,a=1,mode=0,dest])

updates r/g/b/a/mode always, dest only if specified


gfx_blit(input[,preserve_aspect=0,x,y,w,h,srcx,srcy,srcw,srch])

copies input to framebuffer. preserve_aspect=-1 for no fill in pad areas


gfx_fillrect(x,y,w,h)

fills a rectangle


gfx_procrect(x,y,w,h,channel_tab)

processes a rectangle with 768-entry channel table [256 items of 0..1 per channel]


gfx_gradrect(x,y,w,h, r,g,b,a [,drdx,dgdx,dbdx,dadx, drdy,dgdy,dbdy,dady]);


gfx_rotoblit(srcidx, angle [,x, y, w, h, srcx, srcy, w, h, cliptosrcrect = 0, centxoffs=0, centyoffs=0])


gfx_deltablit(srcidx, x,y,w,h srcx,srcy, dsdx, dtdx, dsdy, dtdy, dsdxdy, dtdxdy[, dadx, dady, dadxdy])


gfx_xformblit(srcidx, x,y,w,h, wdiv, hdiv, tab[, wantalpha=0])

tab is wdiv*hdiv*2 table of src points. if wantalpha=1, tab is wdiv*hdiv*3 table of src points including alpha.


gfx_keyedblit(input[,x,y,w,h,srcx,srcy,kv1,kv2,kv3,kv4])

chroma-key blits, using the source color as key. kv1-kv4 meaning depends on colorspace:
YV12/YUY2: kv1 is U target (-0.5 default), kv2 is V target (-0.5 default), kv3 is closeness-factor (0.4 default), kv4 is the gain (2.0 default)
RGBA: kv1 is green-factor (1.0 default), kv2 is blue-factor (-1.0 default), kv3 is offset (-1.0 default), kv4 enables spill removal (1.0 default)


gfx_destkeyedblit(input[,x,y,w,h,srcx,srcy,kv1,kv2,kv3,kv4])

chroma-key blits, using destination color as key. ignores gfx_a and gfx_mode.


gfx_setfont(pxsize[,#fontname, flags)

sets a font, flags can be 'B' for bold, 'I' for italics, 'BI' for bold italics, etc


gfx_str_measure(#string[,w,h])

measures the size of #string, returns width


gfx_str_draw(#string[,x,y,fxc_r,fxc_g,fxc_b])

draw string


rgb2yuv(r,g,b)

utility, converts r,g,b to YUV, does not clamp [0..1]


yuv2rgb(r,g,b)

utility, converts YUV to r,g,b, not clamping [0..1]