The Lumiera Community creates a non linear video editing and compositing FOSS application for Linux/Unix/Posix Operating Systems, suitable for professional and quality oriented work, building on common open source video, sound and GUI toolkits and libraries, providing flexibility and a high degree of configurability and full control of all parameters, but at the same time a smooth workflow which scales well to larger and more complicated editing projects. This Document outlines the Design from some distance, helping people to understand the Ideas behind Lumiera and understand the tools they get to work with. It is aimed for workflow designers any anyone who wants to know how the program works in general.
Lumiera (as seen) from Outer Space
— Summer 2010 —
About this Document
This document is meant to be read electronically, it contains a lot hyper-links between explanations denoted by an arrow →. Lumiera is still in development, we describe here planned features without explicitly tagging them; some things are not worked out in detail yet. Although this document is heavily cross-linked we try to start with a broad overview and work out more detailed things towards the end.
Vision
Lumiera claims to be a professional non-linear video editor. To start with, we should point out that “professional” does not necessarily mean “commercial” or “industrial”. It’s more of an attitude or mindset — doing work seriously, and to be subject to any kind of wider goal, demand, or purpose. When it comes to editing film, this might be artistry, a narration or meaning to convey, a political message or something to show to your audience. Anyhow, for the tools, the editing software used to this end, we can identify several properties and requirements, to be labeled “professional”:
- Reliability
-
Whatever happens, your work must be safe, protected against software glitches and incompatibilities. Ideally Lumiera should be very stable and never crash, in practice even crashes or power outages should not result in lost work.
- Quality
-
If you work with high quality, cinema grade digital video material you want to be sure that you can deliver crisp quality without compromise, throughout the whole workflow to your final product. All rendering must be reproducible to the bit.
- Performance and Productivity
-
Professionals want to get things done, in time, but optionally with control over every aspect. Balancing these goals should be the central concern for workflow design and usability.
- Scalability and Adaptability
-
Projects and budgets differ, hardware advances, Lumiera must scale in different dimensions and use the available resources as best as it can. From small Laptops to multi core computers and Renderfarms.
- Durability
-
Soft and Hardware advances at a fast pace. We must not lock into the current state of technology but being flexible to extend the System without breaking compatibility. Projects you create nowadays with Lumiera should be usable in foreseeable future, at least there needs to be a guaranteed upgrade path.
Fundamental Forces
The Lumiera design is guided by a small number of basic principles. Keeping these in mind will help to understand how actually more interesting things can be built up on that foundation.
Open ended combining of Building Blocks
Lumiera is not so much defined in terms of features — rather it allows to combine basic building blocks. These basic modules, entities or objects each have a distinct type explicitly limiting the connections. Within these limits, any conceivable combination shall be supported without further hidden limitations.
Lumiera is neither a set of Lego bricks, nor is it the business application driven by finite usage stories.
Medium level Abstraction and Project specific Conventions
These building blocks within Lumiera create a moderate level of abstraction; a user may, if desired, directly manipulate through the GUI clips, individual effects, masks, and even the placements → used to stitch the objects together, which is comparatively low-level. On the other hand, these abstractions shield the user from the actual technical details like format conversions and the accessing of individual channels.
To complement this approach, Lumiera does not rely on hard wired, global conventions — rather we allow to build up project specific conventions and rules → to fit the given requirements and preferred working style. To help getting started, Lumiera will ship with a fairly conventional project template and default configuration.
Rendering is Graph Processing
Processing of Video (and audio) data can be generalized as graph processing (more precisely “directed acyclic graphs”). Data flows on the edges of these graphs and is processed in the nodes.
Pulling not Pushing
On a first glance, it looks fairly natural to set up the graphs → as described above and then push data into the system through the input nodes whereas the final result can then be seen soon on the output node. Several multimedia frameworks use this approach. But it has a lot of shortcomings which make it inappropriate for non-linear video editing.
Lumiera instead pulls data though the pipe, that is a request starts at the output node and makes it way up to the inputs. This has certain advantages →, explained later.
The visible Universe
Now its time to take a look at the prelimary Lumiera GUI:
The GUI is a plugin by itself and only one way to work Lumiera, it will become possible to create special-purpose GUIs or control Lumiera in different ways, like a headless rendernode → or frameserver →. Completely script driven interfaces for automated processing are also planned.
The GUI screenshot you see above is faily default as when you start Lumiera up for the first time (the plan is to add a 2nd Viewer to the default configuration). While we support a much more sophisticated screen concept → to adapt to different workplaces and workflows.
Viewer
to be written
Transport Controls
The layout in current gui is rather preliminary — it is not finally decided where transport controls will be integrated; possibly as its own gui element
This are devices either controlled by widgets or by some input device (MIDI, control surface, etc) so their gui may look differently. Either way they connect to a Play Controler xref.. in the core which manages playing and cursor positioning.
thus there will be some gui facility to attach Transport controls to Play Controllers. Transport controls are ganged when they attach to the same Play Controler.
just playing some footage for preview creates a simple internal timeline, no magic here.
Timeline View
hierarchical tracks, not just a list
Format Independent Timeline, one can put anything on the timeline. the busses constrain what kind of data is pulled out and in turn the builder creates a processing graph which does the necessary conversions and stuff.
Busses
The GUI provides a separate bus view, showing the master busses (subgroups) in a manner similar to an audio mixing desk. Any bus is just a means to collect and sum up the output of a specific kind of media (video, audio, number of channels), produced by various processing elements and other busses. Conceptionally, these global busses are considered a part of the timeline
Asset View
Manages all assets available in one project. * source media/footage/soundfiles * prepared clips, known subprojects * all available effects and transitions * internal artefacts like sequences and automation data sets
Dark Matter
to be written coarse overview about things the user does not see but have some contact with, details later…
Now lets take a look under the hood.
Session storage
to be written
Placements
Generic mechanism to stitch together media objects. Any placement may contain a list of conditions how to locate the placed object, examples being time-absolute/relative, relative to another object, or relative to some specific source media frame.
All of the session model contents are attached by placement, forming a large tree. Placements are to be resolved to find out the actual position, output and further locational properties of an object. Missing placement information is inherited from parent placements in the session tree. This causes a lot of relational and locational properties to be inherited from more global settings, unless defined locally at a given object: time reference point, output destination, layering, fade control, audio pan,…
Rendering Engine
to be written rendering…
to be written
rules system to be written
I/O Subsystem
to be written
Configuration
to be written
Plugins/Interfaces
to be written
Rendering Video
to be written
Pulling a Frame
to be written
TODO Consider integrating the following things into the document above
-
plugins
-
timeline
-
pull
-
bus defines rendering format
-
caching
-
frameserver
-
screenconcept / perspectives
-
automation
-
3 layered model
Glossary
The above outline of the design uses a lot of special terms and common termes used with specific meaning within Lumiera. To ease the understanding, we’ve collected a Glossary of common terms.