MediaTomb

Transcoding

This is the place to share transcoding settings and scripts.

The lazy PS3 owner's transcoding howto

Since the PS3 is a really, really crappy picky mediaplayer, you get to spend a lot of time tweaking your mediatomb config to produce an acceptable output stream. The easiest (yet most CPU intensive) solution is to use transcoding on everything: Linux doesn't care what it gets as an input format, it can usually handle it just fine - and if mediatomb produces an mpeg2 stream for every item, the PS3 is happy too.

You can check the howto here: MediaTomb and PS3: The Lazy Man's solution.

Transcoding everything is extremely CPU intensive though, up to a point where it is simply not possible (1080p HD content, etc). It would thus be smarter to not transcode everything, but remux where possible and only transcode those files that cannot be remuxed.

The follow-up article to achieve this: MediaTomb and PS3: The Lazy Man's solution (Part II).

The script provided in this article is able to remux h.264 MKV files on the fly to the PS3, as well as MP4 and DVD. AVI files are transcoded to MPEG-2, with the ability to automatically render subtitles in the stream.

FFmpeg transcoding failing mysteriously

If you're using ffmpeg to transcode things in mediatomb, you probably bumped into this: it just doesn't work, and the transcoding gets stuck after trying. You end up with a dead ffmpeg process that you need to kill -9 afterwards. The reason for this is that mediatomb uses a FIFO for transcoding: the transcoder pipes its stream into the FIFO, and mediatomb reads from that FIFO to serve the stream to the client.

The mediatomb docs mention this: “It is very important that the transcoder is capable of writing the output to a FIFO, some applications, for example ffmpeg, have problems with that”. This is true, since ffmpeg temporarily opens its output file read-only before starting to write to it. If the output file is a (new) FIFO, nothing is in it yet, so the call blocks on this.

This is fixed by using the ”-y” switch, which basically tells ffmpeg to write to the given target, regardless of whether it already exists or not, skipping the read-check.

FFmpeg transcoding too slow

IF FFmpeg is transcoding too slow, which is often the case even on relatively new computers with 720p/1080p content, the ”-intra” option significantly speeds up compression at the expense of file size and quality. This is not necessarily an issue with gigabit/802.11N networks, where you can boost your bitrate to compensate. Intra disables motion estimation completely (you have only I-frames, which means compression is about as good as JPEG compression). This also works if using FFmpeg within VLC.

Default vlcmpeg transcoder not working

If you're using debian or ubuntu, the vlc transcoder in the config won't work.

In newer versions, the mpeg2 encoder was taken out.

To fix this, change vcodec=mp2v to vcodec=mp1v in the <agent command=“vlc”…. line of your config.xml.

This will use the mpeg1 encoder instead of mpeg2.

Transcode RealMedia (.rm) files with mencoder

Since vlc cannot handle RealMedia (or, at least, older ffmpeg cannot, and especially debian based distributions are thus screwed) you'll have to do this with mencoder. The following profile line will transcode your .rm file to a nice mpeg2 file:

<agent command="mencoder" arguments="%in -oac lavc -ovc lavc -of mpeg  
-lavcopts vcodec=mpeg2video:keyint=1:vbitrate=200000:vrc_maxrate=9000:vrc_buf_size=1835 
-mpegopts muxrate=12000 -vf harddup -o %out"/>

For those who are wondering: Yes, this exact same line will also work for any other file you want to transcode, so it can serve as an alternative for those people who are screwed by the mpeg1-restriction in debian mentioned above.

Piggybacking subtitles in your stream

The command above is also very useful for adding those precious .srt subtitle files to your stream! Supposing that every one of your subtitles is named as

  • my_movie.srt
  • my_movie.xx[x].srt (where xx is the language code, 2 or 3 characters)

and there is a corresponding movie named

  • my_movie.ext (where ext can be one of avi, mp4, mpg or mov)

if you then add a separate mimetype for subtitles…

<map from="srt" to="video/subtitle"/>

… and tell your mediatomb to transcode that, using an mencoder profile…

<transcode mimetype="video/subtitle" using="mencoder-srt"/>

… and have your mencoder-srt profile have a script as agent…

<agent command="/usr/local/bin/mediatomb-mencoder-srt" arguments="%in %out"/>

… and have this as mediatomb-mencoder-srt content…

#!/bin/bash
srt="$1"
output="$2"
# filename must be of the form "movie name.XX[X].srt" or "movie name.srt"
base_name="$(echo $srt | sed 's/\..\{2,3\}\.srt$//' | sed 's/\.srt$//')"

extensions="avi mp4 mpg mov"
for ext in $extensions ; do
    input=$base_name.$ext;
    if [[ -e $input ]]; then break; fi
done

exec mencoder "$input" \
-oac lavc -ovc lavc -of mpeg \
-lavcopts vcodec=mpeg2video:keyint=1:vbitrate=200000:vrc_maxrate=9000:vrc_buf_size=1835 \
-vf harddup -mpegopts muxrate=12000 \
-sub "$srt" -font "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf" -subfont-autoscale 2 \
-o "$output"

… guess what? You get a nice mpeg2 stream, with your subtitles embedded. Whoopee!

Range from .flac file

This script is intended to be used to play tracks that are defined as a range in a flac file. The range and the file are specified as a dummy url that is parsed within the script.

Save the following lines as flacNcue2wav.sh and store it to a location in your search path, e.g., /usr/bin/ (Syntax highlighting is omitted because it transforms the code syntax to invalid.)

#!/bin/bash
# flacNcue2wav.sh transcoding script to play a range from a flac file.
INPUT="$1"
OUTPUT="$2"

# parse parameters
SKIP=`echo ${INPUT} | sed 's/http:\/\/cue2flac\/params?.*skip=\([0-9:.]*\).*/\1/'`  
UNTIL=`echo ${INPUT} | sed 's/http:\/\/cue2flac\/params?.*until=\([0-9:.\-]*\).*/\1/'`
FILENAME=`echo ${INPUT} | sed 's/http:\/\/cue2flac\/params?.*filename=\"\(.*\)\".*/\1/'`

# default params
if [ "${SKIP}" == "" ]
then
  SKIP="0:00.00"
fi

if [ "${UNTIL}" == "" ]
then
  UNTIL="-0:00.00"
fi

# if a filename is given, try transcoding
if [ "${FILENAME}" != "" ]
then
  # transcoding command from flac to raw wav with a range
  exec /usr/bin/flac -f -c -d "${FILENAME}" --skip=${SKIP} --until=${UNTIL} > ${OUTPUT} 2>/dev/null
fi

Viewing RAW format pictures

You can use the program 'dcraw' (available from most good linux distributions, nslu2 ipkg etc).

For a canon CR2 file you can choose to use dcraw to transcode the image (slow, even on a top spec machine) or use dcraw to extract and offer the high res embedded jpg (> 1080p).

Presently, with libexif (the default) rather than libextractor builds, no (exif) date is provided to date the file.

Create a 'transcode' dcraw script, I called it '/opt/bin/dcraw-thumb':

#!/bin/sh
exec dcraw -e -c "$1" > "$2"

don't forget to make it executable with a chmod ugo+x script-file-name and then add the following to your config.xml:

  <transcoding enabled="yes">
    <mimetype-profile-mappings>
      <transcode mimetype="image/cr2" using="cr2-jpg"/>
      ...
    </mimetype-profile-mappings>
    <profiles>
 
      <profile name="cr2-jpg" enabled="yes" type="external">
        <mimetype>image/jpeg</mimetype>
        <agent command="/opt/bin/dcraw-thumb" arguments="%in %out"/>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <hide-original-resource>yes</hide-original-resource>
        <buffer size="60000" chunk-size="20000" fill-size="20000"/>
      </profile>
 
     ...
    </profiles>
  </transcoding>

where I have used '…' to indicate the original contents of the sections 'here'.

You will want to add mime-type mappings to the <mappings> section too:

        <map from="cr2" to="image/cr2"/>
        <map from="CR2" to="image/cr2"/>

and there you are. Nikon NEF and Adobe DNG etc are left as an exercise.

DirecTV HR2x Transcoding

Here is the setup information to setup transcoding for the DirecTV HR2x. You MUST use a MediaTomb version later than the 0.11.0 release. The 0.11.0 release (or earlier) will NOT work with the HR2x.

After building and installing MediaTomb, start the MediaTomb server and this will create the .mediatomb directory and within it, an initial config.xml file. Here are modifications needed in config.xml:

config.xml Updated 11/29/08 Cleaned up config.xml and eliminated some transcoding scripts by moving the command line calls into config.xml.

Add these to the default list in <extension-mimetype>

    <map from="ts" to="video/mpeg"/>
    <map from="vob" to="video/mpeg"/>
    <map from="wav" to="audio/wav"/>
    <map from="mpg" to="video/mpeg"/>
    <map from="aac" to="audio/x-aac"/>
    <map from="m4a" to="audio/mp4"/>
 
    <map from="mkv" to="video/x-matroska"/>
    <map from="mov" to="video/x-quicktime"/>

Replace <transcoding> section with this configuration:

  <transcoding enabled="yes">
    <mimetype-profile-mappings>  
      <transcode mimetype="audio/mpeg" using="vlcwav"/>    
      <transcode mimetype="video/x-flv" using="vlcyoutube"/>
      <transcode mimetype="video/mp4" using="transvideo"/>
      <transcode mimetype="video/x-quicktime" using="transvideo"/>
      <transcode mimetype="application/ogg" using="vlcwav"/>
      <transcode mimetype="audio/x-ms-wma" using="vlcwav"/>
      <transcode mimetype="audio/x-ms-asf" using="vlcwav"/>
      <transcode mimetype="audio/x-flac" using="ffmpegwav"/>
      <transcode mimetype="audio/x-aac" using="vlcwav"/>
      <transcode mimetype="audio/mp4" using="vlcwav"/>
      <transcode mimetype="video/x-msvideo" using="transvideo"/>
      <transcode mimetype="video/x-ms-wmv" using="transvideo"/>
      <transcode mimetype="video/mpeg" using="mpeg2trans"/>
      <transcode mimetype="video/x-matroska" using="transvideo"/>
      <transcode mimetype="image/jpeg" using="rescalejpeg"/>
    </mimetype-profile-mappings>
    <profiles>
      <profile name="transvideo" enabled="yes" type="external">
        <mimetype>video/mpeg</mimetype>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <agent command="ffmpegvideo" arguments="%in %out 8300k 256k"/>
        <buffer size="57600000" chunk-size="128000" fill-size="10000000"/>
      </profile>
      <profile name="mpeg2trans" enabled="yes" type="external">
        <mimetype>video/mpeg</mimetype>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <hide-original-resource>yes</hide-original-resource>
        <agent command="ffmpegvideo" arguments="%in %out 8000k 256k"/>
        <buffer size="28800000" chunk-size="512000" fill-size="120000"/>
      </profile>
      <profile name="ffmpegwav" enabled="yes" type="external">
        <use-chunked-encoding>no</use-chunked-encoding>
        <mimetype>audio/wav</mimetype>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <agent command="ffmpegaudio" arguments="%in %out"/>
        <buffer size="1048576" chunk-size="131072" fill-size="262144"/>
      </profile>
      <profile name="vlcwav" enabled="yes" type="external">
	<use-chunked-encoding>no</use-chunked-encoding> 
        <mimetype>audio/wav</mimetype>
        <accept-url>yes</accept-url>
        <first-resource>yes</first-resource>
        <agent command="vlc" arguments="-I dummy %in
           --sout #transcode{acodec=s16l,ab=192,channels=2}:standard{access=file,
                                                      mux=wav,dst=%out} vlc:quit"/>
        <buffer size="512000" chunk-size="32000" fill-size="64000"/>
      </profile>
      <profile name="rescalejpeg" enabled="yes" type="external">
        <mimetype>image/jpeg</mimetype>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>no</accept-ogg-theora>
        <agent command="convert" arguments="-size 1080x720 %in -auto-orient
                                           -resize 112.5%x100% +profile '*' %out"/>
        <buffer size="50000" chunk-size="100" fill-size="100"/>
      </profile>
      <profile name="ffyoutube" enabled="no" type="external">
        <mimetype>video/mpeg</mimetype>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>yes</accept-ogg-theora>
        <agent command="ffmpegyoutube" arguments="%in %out"/>
        <buffer size="14400000" chunk-size="256000" fill-size="80000"/>
      </profile>
      <profile name="vlcyoutube" enabled="yes" type="external">
        <mimetype>video/mpeg</mimetype>
        <accept-url>yes</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>yes</accept-ogg-theora>
        <agent command="vlc" arguments="-I dummy %in
           --sout #transcode{vcodec=mp2v,vb=4096,canvas-width=448,canvas-height=252,
           acodec=mpga,ab=64,samplerate=44100,channels=1}:standard{access=file,
                                                       mux=ts,dst=%out} vlc:quit"/> 
        <buffer size="14400000" chunk-size="256000" fill-size="80000"/>
      </profile>
    </profiles>
  </transcoding>

Here are the shell scripts called from the transcoding profiles (filepaths may differ):

ffmpegaudio

#!/bin/bash

exec /usr/bin/ffmpeg -i "$1" -f wav - > "$2"

ffmpegvideo This script will add letterboxing to the input video to output a 16:9 aspect ratio to the HR2x. note: mediainfo binary must be present

Updated 11/29/08 Should now work for the HR2x feature for auto adjustment of screen size on an HD channel

#!/bin/bash

# This script transcodes various video formats to MPEG2 Video and
# AC3 audio in an MPEG2 Transport Stream for the DirecTV HR2x
#
# FFMpeg and Mediainfo must be installed for this script to work.
#
# Input Parameters: Input, Output, Video Bitrate, Audio Bitrate
#
# Revision 2.00  11/29/2008
# Updated to work with the HR2x auto screen size feature

VBITRATE="$3"
ABITRATE="$4"

width=`/usr/local/bin/mediainfo --Inform=Video\;%Width% "$1"`
height=`/usr/local/bin/mediainfo --Inform=Video\;%Height% "$1"`

audcodec="ac3"

aspect=$(echo "scale=2; ($width/$height)*100" | bc | awk -F '.' '{ print $1; exit; }' ) 

echo $aspect
comp=175
bars=0

if [ "$aspect" -gt "$comp" ]; then
   # needs bars top and bottom
   bars=1
   pad=$(echo "scale=2; (($width/1.75)-$height)/2" | bc | awk -F '.' '{ print $1; exit; }' )
   mod=$(($pad % 2))
   compmod=1
   if [ "$mod" -eq "$compmod" ]; then
      # pad must be an even number
      pad=$((pad+1))
   fi
fi

echo $pad

# Make sure the ffmpeg path is correct
if [ "$bars" -eq 1 ]; then
   # bars top and bottom 
  exec ffmpeg -i "$1" -b ${VBITRATE} -maxrate ${VBITRATE} -minrate ${VBITRATE} \
-bufsize 5097k -bt 380k -padtop ${pad} -padbottom ${pad} -threads 2 -ab ${ABITRATE} \
-acodec ${audcodec} -async 1 -f mpegts -y - > "$2"
else
  # bars left and right
  exec ffmpeg -i "$1" -b ${VBITRATE} -maxrate ${VBITRATE} -minrate ${VBITRATE} \
-bufsize 5097k -bt 380k -padleft 4 -padright 4 -threads 2 -ab ${ABITRATE} \
-acodec ${audcodec} -async 1 -f mpegts -y - > "$2"
fi

ffmpegyoutube This will add padding on the left and right to format for a 16:9 aspect ratio

#!/bin/bash

# bars left and right
exec /usr/local/bin/ffmpeg -i "$1" -r 25 -b 4000k -maxrate 4000k \
     -minrate 4000k -bufsize 1000k -padleft 44 -padright 44 -threads 2 \
     -ac 1 -ab 64k -ar 44100 -acodec mp2 -f dvd -y - > "$2"

For information about configuring config.xml for YouTube content, please refer to the readme.lyx document in the /doc directory from the latest SVN.

vlcaudio This script is needed if you are using vlc 0.9 or later. The formatting has changed. Also note: later versions of vlc will not run as root, so your mediatomb instance needs to run as a non-root user for this to work.

#!/bin/bash

INPUT="$1"
OUTPUT="$2"
AUDIO_CODEC="s16l"
AUDIO_BITRATE="192"
AUDIO_SAMPLERATE="44100"
AUDIO_CHANNELS="2"
FORMAT="wav"

exec /usr/bin/cvlc "${INPUT}" :sout="#transcode{acodec=${AUDIO_CODEC},\
ab=${AUDIO_BITRATE},channels=${AUDIO_CHANNELS}}:\
standard{access=file,mux=${FORMAT},dst=${OUTPUT}}" vlc://quit

If the vlcaudio script is used, update your config.xml as follows

        <profile name="vlcwav" enabled="yes" type="external">             
        <use-chunked-encoding>no</use-chunked-encoding>                   
        <mimetype>audio/wav</mimetype>                                    
        <accept-url>yes</accept-url>                                      
        <first-resource>yes</first-resource>                              
        <agent command="vlcaudio" arguments="%in %out"/>                  
        <buffer size="512000" chunk-size="32000" fill-size="64000"/>      
      </profile>                                                          

DirecTV HR2x Audio Normalization

Contributed by Master63a

Don't you love it when a commercial comes on and your speakers are almost blown? The audio is highly compressed and that is what makes it sound so loud. These scripts

attempt to 'normalize' the audio using sox as the audio processor. Note that sox has extensive audio processing capability such as equalizers, replay gain, reverb, etc and the scripts can easily be modified to use some of them. These are my first real scripts so improvements are welcome.

The first script is designed to process mp3 audio files and will apply compression to local files and suitable audio streams. If a local file has a sample rate that is 44Khz or higher, compression will not be applied so that the script can be used for high quality mp3 audio.

#!/bin/bash 
INPUT="$1" 
OUTPUT="$2" 
AUDIO_CODEC="s16l" 
AUDIO_BITRATE="192" 
AUDIO_SAMPLERATE="44100" 
AUDIO_CHANNELS="2" 
FORMAT="wav" 
 
exec &> /dev/null 
# LOCAL FILES COMPAND IF SAMPLERATE < 44KHZ 
if [ ${1:0:1} = \/ ] 
then 
   mediainfo "$1" | fgrep Sampling > /tmp/samplerate.txt 
   exec </tmp/samplerate.txt 
   read LINE 
   SRATE=$LINE 
   [ ${SRATE:35:2} -ge 44 ] 
   COMPRESS=$? 
   if [ $COMPRESS = 0 ] 
   then  
      exec sox -t .mp3 "$1" -t .wav - >"$2" 
   else  
      echo 'sox companding' 
      exec sox -t .mp3 "$1" -t .wav - >"$2" compand 0.1,0.8 60:-40,-5 -20 -70 .2 
   fi 
# SOX CAN HANDLE SOME OF THIS TYPE 
else if [ ${1:0:5} = 'http:' ] 
   then 
      echo 'sox companding' 
      exec sox -t .mp3 "$1" -t .wav - >"$2" compand 0.1,0.8 60:-40,-5 -20 -70 .2 
   else 
      # BUT NOT THIS TYPE SO USE Boilerjt's VLCAUDIO SCRIPT 
      echo 'vlc' 
      exec /usr/bin/vlc -I dummy "${INPUT}" --sout "#transcode{acodec=${AUDIO_CODEC},\ 
      ab=${AUDIO_BITRATE},channels=${AUDIO_CHANNELS}}:\ 
      standard{access=file,mux=${FORMAT},dst=${OUTPUT}}" vlc:quit >/dev/null 2>&1 
   fi 
fi 

The second script does the same for videos such as youtube and such. If someone can find a way to end the threads more gracefully, please edit this script.

#!/bin/bash 
 
exec &> /dev/null 
# NOT ELEGANT BUT I CAN'T FIND A GOOD WAY TO STOP THE THREADS IF VIDEO IS STOPPED VIA THE HR2X 
rm -f p1.wav 
rm -f p2.wav 
 
# THIS HAS NO AFFECT, PERHAPS WE NEED MEDIATOMB TO SEND A SIGNAL TO END THE SCRIPT 
#trap "{ rm -f p1.wav;rm -f p2.wav }" SIGTERM SIGPIPE EXIT 
 
mkfifo p1.wav 
mkfifo p2.wav 
 
ffmpeg -i "$1" -f wav -vn -ar 44100 -ac 2 -f wav - > p1.wav & \ 
sox -t raw -r 44100 -2 -s -c 2 p1.wav -t wav p2.wav compand .3,.8 -40,-5 -20 -70 .2 & \ 
ffmpeg -i "$1" -s hd720 -b 6000k -threads 2 -ab 192k -i p2.wav -f dvd -map 0:0 -map 1:0 - > "$2"

PlayStation 3 PCM support

Mediatomb version 0.12.0 has support for PCM output for the Play Station 3. This allows streaming audio from flac, ape, ogg etc. to the playstation.

To enable PCM output support get ogg vorbis files, the following section will do the trick.

<mimetype-profile-mappings>
  <transcode mimetype="application/ogg" using="2pcm"/>
</mimetype-profile-mappings>
<profiles>
  <profile name="2pcm" enabled="yes" type="external" >
    <mimetype>audio/L16</mimetype>
    <first-resource>yes</first-resource>
    <accept-url>yes</accept-url>
    <sample-frequency>44100</sample-frequency>
    <audio-channels>2</audio-channels>
    <hide-original-resource>yes</hide-original-resource>
    <agent command="/usr/bin/ffmpeg" arguments="-i %in -ac 2 -ar 44100 -y -f s16be %out"/> 
    <buffer size="1048576" chunk-size="4096" fill-size="1024"/>
  </profile>
</profiles>

The configuration can of course me merged with existing/other mimetype-profile-mappings and profiles sections.

Roku Soundbridge PCM support

Mediatomb version 0.12.0 has support for PCM output for the Roku Soundbridge. This allows streaming audio from flac, ape, ogg etc. to the device.

To enable PCM output support get ogg vorbis files, the following section will do the trick.

      <profile name="oggflac2ROKU-format" enabled="yes" type="external">
        <mimetype>audio/L16</mimetype>
        <buffer size="1048576" chunk-size="131072" fill-size="262144"/>
        <agent command="ogg123" arguments="-d raw -o byteorder:little -f %out %in"/>
      </profile>

Due to the different raw-format compared to the PS3, it's impossible to configure transcoding in a way that it works for both PS3 and the soundbridge at the same time.

Rar Archive Support

Particularly handy for split rar-archives. Works great for PS3.

Add the appropriate parts to the config.xml…

<mappings>
  <extension-mimetype ignore-unknown="no">
    ...
    <map from="rar" to="application/rar"/>
  </extension-mimetype>

  <mimetype-upnpclass>
    ...
    <map from="application/rar" to="object.item.videoItem"/>
  </mimetype-upnpclass>
</mappings>

<transcoding enabled="yes">
  <mimetype-profile-mappings>
    ...
    <transcode mimetype="application/rar" using="rar2mpeg"/>
  </mimetype-profile-mappings>
  <profiles>
    ...

    <profile name="rar2mpeg" enabled="yes" type="external">
      <mimetype>video/mpeg</mimetype>
      <accept-url>no</accept-url>
      <first-resource>yes</first-resource>
      <hide-original-resource>yes</hide-original-resource>
      <agent command="/etc/mediatomb/bin/rar2mpeg" arguments="%in %out"/>
      <buffer size="28800000" chunk-size="512000" fill-size="12000"/>
    </profile>
  </profiles>
</transcoding>

With the following in /etc/mediatomb/bin/rar2mpeg

#!/bin/sh

INPUT="$1"
OUTPUT="$2"

exec /usr/bin/unrar p -inul "$INPUT" | \
/usr/bin/vlc -I dummy - --sout="#transcode{venc=ffmpeg,vcodec=mp2v,vb=4096,fps=25}:standard{access=file,mux=ps,dst=$OUTPUT}vlc:quit" >/dev/null 2>&1

dvr-ms Windows MCE Records Support

Partial support to play dvr-ms Files directly. Works with Linksys Kiss 1600. Works only with filenames without capital Letters. (i dont know why)

The dvr-ms File looks like an ASF File. It has an Video Stream (mpeg2) and an Audio Stream (mp3). So we can use direct Stream copy in ffmpeg.

Add the appropriate parts to the config.xml…

<mappings>
  <extension-mimetype ignore-unknown="no">
    ...
    <map from="dvr-ms" to="video/x-dvr-ms"/>
  </extension-mimetype>
</mappings>

<transcoding enabled="yes">
  <mimetype-profile-mappings>
    ...
    <transcode mimetype="video/x-dvr-ms" using="ffmpegmce"/>
  </mimetype-profile-mappings>
  <profiles>
    ...
   <profile name="ffmpegmce" enabled="yes" type="external">
        <mimetype>video/mpeg</mimetype>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>yes</accept-ogg-theora>
        <agent command="ffmpeg-mce" arguments="%in %out"/>
        <buffer size="57600000" chunk-size="128000" fill-size="1000000"/>
     </profile> 
  </profiles>
</transcoding>

With the following in /usr/bin/ffmpeg-mce an direct Streamcopy of the dvr-ms file is used for a quick transformation.

#!/bin/sh
exec /usr/bin/ffmpeg -i "$1" -vcodec copy -acodec copy  -f dvd -y  "$2"

Script to perform various transcoding routines

Although it's possible to have mediatomb call 'n' number of programs and/or scripts to perform various transcoding routines, such an approach can become hard to debug and maintain. A single script that can perform 'n' number of transcoding routines can be more useful. The following script can be used to perform various transcoding routines.

#!/bin/sh
 
# This script is used by MediaTomb for transcoding support.
 
# Basic function that will transcode any input file into the target type
# 'dvd' ffmpeg supports
video_common() {
    local input="$1"
    local output="$2"
    ffmpeg -i "$input" -target dvd -y "$output"
}
 
# Basic function that will transcode any input file into the wav format
audio_common() {
    local input="$1"
    local output="$2"
    ffmpeg -i "$input" -f wav -y "$output"
}
 
# Function to transcode matroska files to dvd compatible format
transcode_matroska() {
    local input="$1"
    local output="$2"
    local alang="$3"
    local slang="$4"
 
    # Default to english if languages are not set
    [ "$alang" ] || alang="eng"
    [ "$slang" ] || slang="eng"
 
    # Use mencoder if available, else fall back to using ffmpeg.
    # Transcoding using ffmpeg will not include subtitles.
    if which mencoder >/dev/null 2>&1; then
        mencoder "$input" -oac lavc -ovc lavc -of mpeg -mc 0 -noskip \
            -lavcopts vcodec=mpeg2video:vbitrate=6000:acodec=ac3:abitrate=448 \
            -vf harddup -alang "$alang" \
            -slang "$slang" -font 'Bitstream Vera Sans' \
            -o "$output"
    else
        ffmpeg -alang "$alang" -slang "$slang" -i "$input" -target dvd \
            -y "$output"
    fi
}
 
# Our "main" function below.
 
USAGE="
This script is used by MediaTomb for transcoding support. It can also serve as
a script to transcode various files in a format suitable for streaming directly.
 
Synopsis:
 mediatomb-transcode [TRANSCODE FUNCTION OPTION] [GENERIC OPTIONS]
 
Generic Options:
 -h, --help                 Display this help message.
 -i, --input                Input file that is meant to be transcoded.
 -o, --output               Output file that will be read back by MediaTomb.
 --audio-lang               Specify ISO 639 language code to use for audio.
 --subtitle-lang            Specify ISO 639 language code to use for subtitles.
 
Transcode Function Options (one is required):
 --video-common             Perform generic video transcoding.
 --audio-common             Perform generic audio transcoding.
 --transcode-matroska       Used in transcoding matroska files.
"
 
while [ "$#" -gt "0" ]
do
    case "$1" in
        -i|--input)
            INPUT="$2"
            shift; shift;
            ;;
        -o|--output)
            OUTPUT="$2"
            shift; shift;
            ;;
        --video-common)
            USE_VIDEO_COMMON=1
            shift
            ;;
        --audio-common)
            USE_AUDIO_COMMON=1
            shift
            ;;
        --transcode-matroska)
            USE_TRANSCODE_MATROSKA=1
            shift
            ;;
        --audio-lang)
            ALANG="$2"
            shift; shift;
            ;;
        --subtitle-lang)
            SLANG="$2"
            shift; shift;
            ;;
        -h|--help|*)
            echo "${USAGE}"
            exit 1
            ;;
    esac
done
 
# Perform specified function
if [ $USE_VIDEO_COMMON ]; then
    video_common "$INPUT" "$OUTPUT"
elif [ $USE_AUDIO_COMMON ]; then
    audio_common "$INPUT" "$OUTPUT"
elif [ $USE_TRANSCODE_MATROSKA ]; then
    transcode_matroska "$INPUT" "$OUTPUT" "$ALANG" "$SLANG"
else
    # Must specify one transcoding function
    echo "${USAGE}"
    exit 1
fi

More functions and options can be added to make use of more transcoding routines, such as the routines in the previous sections of this page.

Afterwards, to make use of this script, ensure that the agent command is set to 'mediatomb-transcode' and the correct arguments are supplied. The following can be set as the 'transcoding' section of the config.xml file.

  <transcoding enabled="yes">
    <mimetype-profile-mappings>
      <transcode mimetype="video/x-flv" using="video-common"/>
      <transcode mimetype="application/ogg" using="audio-common"/>
      <transcode mimetype="audio/x-flac" using="audio-common"/>
      <transcode mimetype="video/divx" using="video-common"/>
      <transcode mimetype="video/x-matroska" using="transcode-matroska"/>
    </mimetype-profile-mappings>
    <profiles>
     <profile name="video-common" enabled="yes" type="external">
        <mimetype>video/mpeg</mimetype>
        <accept-url>yes</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>yes</accept-ogg-theora>
        <agent command="mediatomb-transcode" arguments="-i %in -o %out --video-common"/>
        <buffer size="14400000" chunk-size="512000" fill-size="120000"/>
      </profile>
      <profile name="audio-common" enabled="yes" type="external">
        <mimetype>audio/wav</mimetype>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>no</accept-ogg-theora>
        <agent command="mediatomb-transcode" arguments="-i %in -o %out --audio-common"/>
        <buffer size="1048576" chunk-size="131072" fill-size="262144"/>
      </profile>
      <profile name="transcode-matroska" enabled="yes" type="external">
        <mimetype>video/mpeg</mimetype>
        <accept-url>yes</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>yes</accept-ogg-theora>
        <agent command="mediatomb-transcode" arguments="-i %in -o %out --transcode-matroska"/>
        <buffer size="14400000" chunk-size="512000" fill-size="120000"/>
      </profile>
    </profiles>
  </transcoding>

10 June 2010 - French FAI "Free" freebox device Linux config & transcoding settings

( should I wrote this in french :)

Preambule

Why mediatomb ? I found others ( xbmc, mythtv ) bloated. In mediatomb, a few files does the stuff. And for getting a multimedia home player ( especially freebox ) to work, that is great.

Getting it working was long & difficult, because of :

  • The specific multimedia ( and undocumented ) freebox device capabilities
  • The multimedia jungle og codecs, container formats, mime types, …
  • I encountered strange behaviours in Web UI, maybe javascript bugs ( this expression is redondant, no ? ). But nothing really serious. Mediatomb is great software.

BUT IT WORKED.

1) config

freebox v5 HD

debian lenny 503 ( amd64 )

mediatomb-0.12.1 from sources (compiled & runned inside anjuta - linux is very cool now ! )

ffmpeg, libavcodec, … from debian-multimedia lenny main

2) Freebox container formats (headache):

Freebox only support avi, mkv, ts container formats. I have succesfully tested, for each container formats, the following combination of codecs :

  • avi + xvid + ac3
  • mkv + xvid + ac3
  • ts + mpeg2 + ac3

3) TechNow!-ledge : difference between streaming files and streaming transcoded files

When mediatomb streams a file to the device without transcoding, he has access to the whole file, since the file resides on your hard drive. Sniff analysis shows a HTTP 'PARTIAL-CONTENT' conversation with the device.

When mediatomb streams a file to the device with transcoding, he has _NOT_ access to the whole transcoded file, since the file is beeing transcoded while sending it. The transcoded file never entirely exist on your hard drive. Sniff analysis shows a HTTP 'CHUNCK CONTENT' conversation with the device.

4) _THE_ problem

Some container formats stores information at the end of the file. So to play the file, the device needs to seek to the end of the file.

  • avi store infos at the end of the file.
  • mkv store infos at the end of the file.
  • ts does _NOT_ store infos at the end of the file.

So when you transcode a file for freebox, you MUST use the ts format. Else the freebox, seeing a 'HTTP CHUNCK CONTENT', knows that she cant access to the end of the avi or mkv stream, and dont even try. Nothing happens.

5) Transcoder

I choosed ffmpeg, because passing specific arguments to ffmpeg from mencoder sometimes failed ( Video transcoding is a pain. Will someone one day write an encoding-options generator ? ), and vlc refused to play too many files - especially flv files.

ffmpeg tricks :

  • I used the -y ffmpeg option. It maybe unuseful. But it works with it.
  • ffmpeg cant ouput to a pipe ; you will get the infamous message : Unable to find a suitable output format for 'pipe:' ( I think it depends on your version ; the problem is that ffmpeg choose container formats from output file extension ; even if you rename the pipe with f.a. mkfifo myfifo.ts, it does not work ; and specifying -f ts is not recognized in my version of ffmpeg ! ), so I used a trick. Instead of sending transcoded stream to the mediatomb fifo pipe, I simply store it in file. Just before transcoding, I create a shell job that send the content of this file to the mediatomb fifo pipe ( how ? with : tail -c 1024 -f ffmpeg_output_file > mediatomb_fifo_file ). So we do : file.flv –> file.ts –> mediatomb fifo. In fact we stream two times, instead of one.

6) Mime types ( headache )

Freebox mime types

Freebox understand the following mimetypes :

  • x-msvideo (that I used for .avi files)
  • x-matroska (that I used for .mkv files)
  • x/mp2t (that I used for .ts files)
Mediatomb mime types

For beeing able to add a file to the database, its extension must be mapped to a mime type, and that mime type must be mapped to an upnp class.

Example :

      <extension-mimetype ignore-unknown="no">
        <map from="abc" to="video/abc"/>
      </extension-mimetype>
 
      <mimetype-upnpclass>
        <map from="video/*" to="object.item.videoItem"/>
      </mimetype-upnpclass>

will let you add to your database .abc files

7) We' re ready

( we will focus ourselves on .flv files for the transcoding process )

mediatomb.transcode.bash : launch a job to stream the transcoded ffmpeg outputed file to mediatomb fifo, then start transcoding

#!/bin/bash
 
INPUT="$1"
MEDIATOMB_FIFO="$2"
TS_TRANSCODED_FILE="dummy.ts"
 
function transcode_video_ffmpeg_mpeg2_ac3_ts
{
	# Create the ts buffer file
	# Note : Emptying the ts buffer file maybe make unuseful the use of the ffmpeg -y option.
	rm	-f "${TS_TRANSCODED_FILE}"
	touch	"${TS_TRANSCODED_FILE}"
 
	# Start to stream the ts buffer file to the mediatomb fifo pipe in background
	tail 	-q -c 1024 -f "${TS_TRANSCODED_FILE}" > "${MEDIATOMB_FIFO}" &
 
	# Start to transcode, ouputing to the ts buffer file
	# Note : I know I can replace this bash process by the ffmpeg process using 'exec ffmpeg -y ...'
	ffmpeg -y -i "${INPUT}" -vcodec mpeg2video -sameq -r 25 -s ega -acodec ac3 -ar 44100 -ac 2 "${TS_TRANSCODED_FILE}"
}
 
transcode_video_ffmpeg_mpeg2_ac3_ts

mediatomb config.xml customization for the freebox :

...
 
<mappings>
 
	<extension-mimetype ignore-unknown="no">
 
		<!-- ----- FOR NON-TRANSCODED FILES (files that freebox can play) ----- -->
 
		<!-- The freebox will only display .mkv, .avi, and .ts files. -->
		<!-- But these file _MUST_ also be presented to the freebox with a specific mime type. -->
		<!-- Here they are : -->
		<map from="mkv"		to="video/x-matroska"/>
		<map from="ts"		to="video/mp2t"/>
		<map from="avi"		to="video/x-msvideo"/>
 
		<!-- ----- FOR TRANSCODED FILES (files that the freebox cant play) ----- -->
 
		<!-- We define a mime type just for binding it to a transcoding profile. -->
		<map from="flv" to="video/x-flv"/>
 
	</extension-mimetype>
 
	<mimetype-upnpclass>
		<map from="audio/*" to="object.item.audioItem.musicTrack"/>
		<map from="video/*" to="object.item.videoItem"/>
		<map from="image/*" to="object.item.imageItem"/>
	</mimetype-upnpclass>
 
	<mimetype-contenttype>
	...
 
	<!-- Nothing freebox-specific to add here -->
 
	</mimetype-contenttype>
 
 
</mappings>
 
...
 
<transcoding enabled="yes">
 
    <mimetype-profile-mappings>
      <transcode mimetype="video/x-flv" using="freebox-ts"/>
    </mimetype-profile-mappings>
 
    <profiles>
		<profile name="freebox-ts" enabled="yes" type="external">
 
		<!-- freebox compatible mimetype for ts is 'video/mp2t' -->
		<mimetype>video/mp2t</mimetype>
		<accept-url>yes</accept-url>
 
		<!-- Hide true name of the file, else freebox wont display it. 	-->
		<!-- Maybe is the <first-resource>no</first-resource> tag unuseful. -->
		<first-resource>no</first-resource>							
		<hide-original-resource>yes</hide-original-resource>		
 
		<!-- transcode	-->
		<agent command="mediatomb.transcode.bash" arguments="%in %out"/>
 
		<!-- These settings works well for me ; find yours ! -->
		<buffer size="1000000" chunk-size="10000" fill-size="200000"/>
		</profile>
	</profiles>
 
</transcoding>
 
...

Thats all folks !

Easy Transcoding for WDTV (and others)

The WDTV is a nifty little box that can play almost everything out of the box, except for some somewhat obscure video codecs, like MSMPEG4v2 (which my beloved MST3K collection is encoded in), Flash Video (FLV), and Apple Lossless Audio (m4a) files. Since it would be a pain, not to mention a quality hit to permanently re-encode those, I wanted to use MediaTomb to transcode them on the fly.

Unfortunately, none of the default settings or suggestions documented above seemed to work properly for me, so I had to look around for myself.

And I ended up with my two 'go-to' programs for video and audio encoding; ffmpeg and sox.

I made it a goal to keep things simple and not use external scripts for the encoding and came up with the following, for encoding AVIs with MSMPEG4v2 video, FLV, and m4a files to something the WDTV could handle.

 <transcoding enabled="yes">
    <mimetype-profile-mappings>
      <transcode mimetype="video/x-flv" using="tompeg"/>
      <transcode mimetype="audio/mp4" using="topcm"/>
      <transcode mimetype="video/x-msvideo" using="tompeg"/>
    </mimetype-profile-mappings>
    <profiles>
      <profile name="topcm" enabled="yes" type="external">
        <use-chunked-encoding>no</use-chunked-encoding>
        <mimetype>audio/l16</mimetype>
        <accept-url>no</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>no</accept-ogg-theora>
        <agent command="sox" arguments="%in -t s16 %out"/>
        <buffer size="1024000" chunk-size="10240" fill-size="10240"/>
      </profile>
      <profile name="tompeg" enabled="yes" type="external">
        <mimetype>video/mpeg</mimetype>
        <use-chunked-encoding>yes</use-chunked-encoding>
        <avi-fourcc-list mode="process">
          <fourcc>MP42</fourcc>
        </avi-fourcc-list>
        <accept-url>yes</accept-url>
        <first-resource>yes</first-resource>
        <accept-ogg-theora>yes</accept-ogg-theora>
        <agent command="ffmpeg" arguments="-i %in -acodec mp2 -ac 2 -ab 384k -ar 44100 -g 15 -b 2500k -bt 300k -async 1 -y -f dvd %out"/>
        <buffer size="2048000" chunk-size="102400" fill-size="204800"/>
      </profile>
    </profiles>
  </transcoding>

The 'topcm' profile takes pretty much any audio format in, and converts it to raw PCM for the WDTV. Sox is a wonderfully versitle audio conversion program, and handles everything from Apple Lossless to mp3 to old Sun .au audio files.

'tompeg' takes any video format that ffmpeg can decode, and converts it to an 2500kb/s MPEG-2 stream with MPEG2 audio.

This configuration works great with my venerable and slow PPC Mac Mini, and can easily convert a 480×360, 29.97fps avi to mpeg2 in real time.

 
transcoding/transcoding.txt · Last modified: 2012/03/01 06:32 by zabby