MP4BOX HELP

A rather complete description of MP4Box functionalities and various switches.

MPEG-4 SYSTEMS

Documentation of GPAC specific handling of MPEG-4 scene description textual formats: BT and XMT-A.

NHNT & NHML

Generic multiplexing languages to manipulate any media format with GPAC and MP4Box.

TIMED TEXT

Description of 3GPP/MPEG-4 Timed Text tracks for subtitles, tickers ...

ISMA E&A - OMA DRM

Encrypting and decrypting content according to ISMA E&A and OMA DRM 2.0 specifications.

BT Format

BT stands for BIFS Text and is an exact textual representation of the MPEG-4 BIFS scene. Its syntax is the same as the VRML/X3D (.wrl and .x3dv files) ones for the scene description part, and it has been extended for other MPEG-4 tools (OD, OCI, IPMP).

You will find plenty of example BT files in the tutorial and regression test suite, and we strongly recommend using these to get more familiar with the BT syntax.

The BT language has been originally developped at ENST as the textual format of the late MP4Tool. The format is still enhanced at ENST for GPAC needs, but we try not to mess up too much with it to keep VRML compatibility.

There are 3 major parts in a BT file:

• The root scene, made of a collection of PROTO nodes if desired, a single top level node and a list of routes for interaction if needed. This is the part common to BT and VRML formats.
• The InitialObjectDescriptor, describing the streams that must be opened when opening the MPEG-4 scene. It usually contains a BIFS stream description and an OD stream description when visual/audio media are present in the scene.
• A succession of modification to the scene and their associated timing.

MP4Box specific BT syntax

The syntax of BIFS/OD commands in BT has been enhanced to enable animation stream and scalable description encodings.
The new elements are placed at the access unit declaration 'AT TIMING' element:

• 'RAP' element: specifies the following access unit is a random access points. This is needed because MP4Box cannot currently compute the scene random access state per stream. Note however that the first access unit of any systems stream is considered as a random access point
• 'IN' element: specifies the following access unit happens in the stream of given ID.

For example, RAP AT 1000 IN 20 { ... } means that the access unit is a random access point, its timing is 1000 (in stream timescale) and it happens in stream whose ES_ID is 20.

The syntax of the 'AT TIMING' element has been extended to support differential timing to enable authors to specify relative time between commands rather than absolute timing. The differential timing is signaled by using a capital D before the timing itself.

For example, AT D2000 { ... } means that this access unit occurs 2000 time ticks after the preceding access unit.

Importing media with MP4Box and BT

MP4Box uses a specific descriptor for stream importing called MuxInfo. This descriptor is not normative (although used by MP4Tool and all tools from MPEG-4 Systems Reference Software). It is never encoded in the file. The modifications made to this descriptor are backward compatible with other tools using this descriptor. The current syntax is:

MuxInfo {

fileName

streamFormat

GroupID

startTime

duration

frameRate

useDataReference

noFrameDrop

SBR_Type

compactSize

textNode

fontNode

}

Semantics :

fileName : specifies location of stream to import. Optional for BIFS/OD streams, required for others. Supported formats are the same as those supported by MP4Box:

• AVI, MPEG: syntax "src_filename" if only one video in movie, otherwise "src_filename#audio", "src_filename#video".
• MP3, AAC-ADTS, JPG, PNG, SRT, SUB: syntax "src_filename".
• NHNT: syntax "src_filename" where fileName is either the ".nhnt" or the ".media" file of the NHNT source.
• NHML: fileName must be the NHML source file.
• IsoMedia (MP4/3GP): syntax "src_filename" if only one track in file, "src_filename#trackID" where TrackID is the ID of the track to import from src_filename.

streamFormat : optional, one of "AVI", "MP3", "JPEG", "PNG", "MP4", "NHNT", "SRT" depending on input type. This should not be needed as MP4Box uses the file extension to figure out the media format, it is kept for old BT compatibility.

GroupID : optional integer, specifies the multiplexing order in the final mp4. Media tracks are interleaved by groups, the group with the lowest ID being written first to disk. Using groups may greatly improve http streaming of the file.

startTime : optional integer, specifies the DTS/CTS of the first sample - by default the first sample imported has CTS/DTS 0. Expressed in milliseconds.

duration : optional integer, specifies run-time of media data (from source start) to import. Expressed in milliseconds.

Note : When creating OCR tracks in MP4, you must use the muxInfo.duration to specify the desired duration of the OCR track, otherwise the OCR track won't have any associated duration and will never stop (looping not possible).

useDataReference : optional boolean, specifies that media data shall not be copied in the final MP4 but only referenced. This may not be supported depending on input data framing. cf '-dref' option in MP4Box.

frameRate : overrides the source media framerate when possible (same as MP4Box -fps).

noFrameDrop : optional boolean for AVI import only, specifies that video shall be imported at constant frame rate, eg non coded frames in AVI file shall be kept in MP4. cf '-nodrop' option in MP4Box.

SBR_Type : optional string for AAC-SBR import only, either "implicit" or "explicit" - cf MP4Box -sbr and -sbrx options.

compactSize : optional string, either "TRUE" or "FALSE", indicating if sample sizes should be stored in a compact way or not.

textNode, fontNode : required string identifier for SRT importing, ignored for other streams. SRT importing is done by creating a BIFS animation stream carrying the text and fonts modifications (sample available in regression test suite).

• The textNode shall be the DEF identifier of the text node to modify (it can be a proto, but then it MUST have an MFString field named "string")
• The fontNode may be ignored. If set, it shall be the DEF identifier of the font node to modify (it can be a proto, but then it MUST have an SFString field named "style")

XMT Format

XMT is the official textural description of MPEG-4 scenes. It is part of ISO/IEC 14496-11, and is quite similar to the X3D XML language. For more details on the XMT syntax, please have a look at the tutorial.

Importing media with MP4Box and XMT

There is no standard element in XMT-A allowing to describe multiplexing parameters for MP4. In order to do this MP4Box uses an XML representation of MuxInfo in the StreamSource sub elements (encoding hints) called MP4MuxHints.

The complete syntax is:

<StreamSource url="filename" >

<MP4MuxHints GroupID="..." startTime="..." duration="..." useDataReference="..." noFrameDrop="..." SBR_Type="..." frameRate="..." compactSize="..." textNode="..." fontNode="..." />

</StreamSource>

The streamFormat parameter is not represented in XMT-A.

Note When decompressing an mp4 file to bt (XMT-A), a MuxInfo (MP4MuxHints) is automatically generated for all streams other than BIFS and OD in order to keep track of media location in the original MP4. You can then simply modify and reencode the BT or XMT file without specifying the media data location.

MPEG-4 Scene Commands

We will now review the syntax of MPEG-4 scene commands in both BT and XMT-A formats. Please remember that BT and XMT languages are case sensitive.

Command declaration

BT format

RAP AT time IN esID { .... }

RAP and IN esID can be omitted most of the time.
XMT format

<par begin="time" atES_ID="esID" isRAP="yes" > ... </par>

isRAP and atES_ID can be omitted most of the time. isRAP is a GPAC extension.

Replacing a simple field

BT format

REPLACE nodeName.fieldName BY newValue

XMT format

<Replace atNode="nodeName" atField="fieldName" value="newValue" />

Replacing a SFNode field

BT format

REPLACE nodeName.fieldName BY NodeDeclaration {... }

XMT format

<Replace atNode="nodeName" atField="fieldName" >

<NodeXXX />

</Replace>

Note that the new node can be DEF'ed, or that a null node may be specified (NULL).

Replacing a value in a multiple field

BT format

REPLACE nodeName.fieldName[idx] BY newValue

XMT format

<Replace atNode="nodeName" atField="fieldName" position="idx" value="newValue" />

For XMT-A, idx can also take the special values 'BEGIN' and 'END'.
Replacement of a node in an MFNode field is the combinaison of both syntax

Replacing a multiple field

BT format

REPLACE nodeName.fieldName BY [value1 ... valueN]

REPLACE nodeName.fieldName BY [Node { ... } ... Node { ... }]

XMT format

<Replace atNode="nodeName" atField="fieldName" value="value1 ... valueN" />

<Replace atNode="nodeName" atField="fieldName" >

<NodeXXX>...</NodeXXX>

<NodeXXX>...</NodeXXX>

</Replace>

Replacement of a node in an MFNode field is the combinaison of both syntax

Deleting a node

BT format

DELETE nodeName

DELETE nodeName.fieldName

XMT format

<Delete atNode="nodeName" />

<Delete atNode="nodeName" atField="fieldName" />

Deleting a value in a multiple field

BT format

DELETE nodeName.fieldName[idx]

XMT format

<Delete atNode="nodeName" atField="fieldName" position="idx" />

For XMT-A, idx can also take the special values 'BEGIN' and 'END'.

Inserting a simple value in a multiple field

BT format

INSERT AT nodeName.fieldName[idx] newValue

APPEND TO nodeName.fieldName newValue

XMT format

<Insert atNode="nodeName" atField="fieldName" position="idx" value="newValue" />

For XMT-A, idx can also take the special values 'BEGIN' and 'END'.

Inserting a node in a node list field

BT format

INSERT AT nodeName.fieldName[idx] Node { }

APPEND TO nodeName.fieldName Node { }

XMT format

<Insert atNode="nodeName" atField="fieldName" position="idx" >

<NodeXXX>...</NodeXXX>

</Insert>

For XMT-A, idx can also take the special values 'BEGIN' and 'END'.

Replacing a route

BT format

REPLACE ROUTE routeName BY nodeName1.fieldName1 TO nodeName2.fieldName2

XMT format

<Replace atRoute="routeName">

<ROUTE fromNode="nodeName1" fromfield="fieldName1" toNode="nodeName2" toField="fieldName2" />

</Replace>

Inserting a route

BT format

INSERT ROUTE nodeName1.fieldName1 TO nodeName2.fieldName2

XMT format

<Insert>

<ROUTE fromNode="nodeName1" fromfield="fieldName1" toNode="nodeName2" toField="fieldName2" />

</Insert>

MP4Box tips and tricks with BT and XMT

Minimal Stream Descriptors for MP4Box (BT and XMT)

When encoding a BIFS or OD ES_Descriptor, MP4Box must find at least:

• The decoderConfigDescriptor with the right streamType set

When encoding an interaction stream descriptor, MP4Box must find at least:

• The decoderConfigDescriptor with the right streamType set
• The UIConfig descriptor with the deviceName field. Note however that most of the time you will need to specify the ES_ID and the OCR_ES_ID of this descriptor to make sure events are not tied to any timeline.

When encoding a regular ES_Descriptor, MP4Box must find at least:

• The MuxInfo descriptor with the fileName (StreamSource in XMT) set.

For systems streams, if the SLConfigDescriptor is not found, MP4Box uses a stream timescale of 1000, otherwise it uses SLConfigDescriptor.timestampResolution as stream timescale.

When the InitialObjectDescriptor is not found in the BT file, MP4Box will create one for you.

Non-linear Parsing

MP4Box can perform non-linear parsing of text files (BT,XMT,VRML,X3D/XML), in other words it understands usage of a node before its definition. This greatly simplifies content authoring, in terms of complexity (for example, 2 conditionals referencing each-other) and readability (you don't have to declare things at specific places). When encoding, node declarations are put back before node referencing (thus the decoded file will not look the same as the original one).

Multiple DEF handling

MP4Box can handle redefinition of nodes with the same DEF identifier (ex, "DEF AC AudioClip" and "DEF AC MovieTexture"). However when doing so, the nodes don't have the same binary IDs and only the first DEFed node can be safely used for field replacement. This feature should only be used with care to replace a whole node.

Forcing binary IDs

You can make MP4Box use your own binary identifiers for nodes and routes by using the syntax NXX for nodes or RXX for routes, where XX is the desired binary identifier (>=0). In case a node in the scene already has the same binary ID, its ID is changed to a non-conflicting value and MP4Box will print a warning message indicating a node ID has been changed.

HTML Color Codes (BT only)

MP4Box can use HTML color codes in BT instead of the regular 3 floats SFColor. Since '#' is a comment character in BT/WRL, HTML color codes are signaled by the '$' character, for example emissiveColor $FFAAB4.

Simple Macro processor (BT only)

MP4Box can use simple macros in BT in order to help authoring. A BT (or WRL) file with such macros cannot be understood by any other tools than MP4Box!.

The macro must be defined on a single line as follows:

#define MACRO_NAME REP LAC E MENT STR ING

The BT parser will simply replace any occurence of the macro by its value.

Here is an example of a BT macro:

#define MYAPP Appearance { material Material2D { emissiveColor 1 0 0 filled TRUE} }

#define MYCOORDS 200 20

Transform2D {

translation MYCOORDS

children [

Shape {

appearance MYAPP

geometry Rectangle { size 20 20 }

}

]

}