Wednesday, August 6, 2008

syBitmap finished! Now what?

The dev in charge of the Video playback controls is going to be away this week. So maybe it's time to start designing the Renderer API.

Which functions will it have? How will it tell the codecs to read a file? How to handle video and audio sync? How to handle the threads?

Too many questions, any help appreciated. Thanks.


JeCh said...

Hi, I just recently discovered your project and I think it has great future. I have a few comments, which I hope might interest you.

I'm not a good programmer, but I'm currently working on a PyGTK video transcoding application. It is rather a GUI for Avisynth and a lot of CLI applications. My knowledge about video processing is probably much better then programming skills.

So I hope I can help with some ideas in this area. Basically I think you should decide what you want to support and where you can get support for it. To read most multimedia files you can rely on FFmpeg. It can decode almost anything. It can also export to most common formats except of H.264/MPEG-4 AVC, but we have the great x264 encoder for this task. I think that including ffmpeg is a very good idea for the start, other input/output frameworks (GStreamer, VfW, DirectShow) might be implemented later.

I think you might find a very good source of information at doom9 forums ( There are plenty of knowledgeable people, many of them are also very good programmers working on different opensource projects.

So my advice is: Have a look at FFmpeg, it should answer most of your questions. And don't forget about XviD, x264, Aften, lame, MP4Box, MKVToolnix (libmatroska), Ogg libraries and other (L)GPL projets.

Also I have one comment to your Mission Statement. There's no word about quality. And I think that this is a very important point. The opensource word offers very high quality audio/video codecs, (de)muxers and filters. One of the key features should be something called "smart rendering" or "direct stream copy". This means that if the output settings and input files have the same parameters, the audio and video won't be compressed again. Here are some arguments why this is so important:

1) If you don't recode the parts which don't need to be converted you will get better quality. Reencoding degrades quality quite a lot. It becomes more significant now that many camcorders use MPEG-2 and MPEG-4.

2) You will also get much better speed. It becomes much more important with AVCHD camcorders. For most today's processor it is not possible to decode such video in realtime. Now imagine what it takes to decode and encode the video and compare it to simply copying the data.

All this should be possible to solve quite easily with the help of FFmpeg. Just find the parts which don't need to be converted and encode only the those that need it. Then join it all together. You might have a look at Womble MPEG Wizard which implements this nicely. Also SONY Vegas has this feature for a limited set of formats and btw. this editor has a very nice user interface IMO.

Also please try to find a way to use Avisynth and VirtualDub filters at least on Windows. It should be possible to port Avisynth filters for Linux since they have no GUI. I think that Avidemux developers ported some of them already. Also would it be possible to support FreeFrame ( effects?

I hope my post was at least a little informative to you. I wish you a lot of luck with your effort and I really hope you will succeed. I have a very similar opinion on most of the points you wrote in your Mission Statement.

Best regards,

rick_777 said...

Vlada: Thanks a lot for your entry!

For the codecs, I've ruled out FFMPEG due to the horrible codebase and lack of updates in that project. So I'm going for gstreamer.

About the direct stream copy, I know that feature from the old Virtualdub days. Unfortunately this tool is for video creation and not simple transcoding. But if Sony Vegas does it, then I guess we'll be able to do it yet. Perhaps by the time we're quite advanced we'll think of how to do it.

Please add it to the feature requests at (you'd need to get an account, unfortunately. It's necessary because we don't want to see a feature request called "cheap mortgage rates" or something :P

Regarding virtualdub filters, I can't guarantee anything, but one of the team members made an avisynth plugin for Adobe Premiere. It's probable that we'll be able to do something similar for Saya.

Finally, regarding Freeframe, guess what? That was my choice of effect plugins provider ;-)

Stay in touch.

rick_777 said...

P.S. How did you find out about the project? I'm curious.

JeCh said...

I'll start from the end: I found your project on SourceForge.

According to GStreamer/FFmpeg I think that GStreamer is a good choice. As a mostly Windows user I don't have much experience with it, but I think it is a good framework. But I'm not sure how far the Windows port of GStreamer is.

Anyway I think even GStreamer is using libavformat and libavcodec (parts of FFmpeg) to (de)mux and (de)code multimedia. I don't think there is a lack of updates to FFmpeg. These guys are doing a great job IMO.

I'm glad to hear all that good new from you regarding plugins/filters. According to my "Direct Stream Copy" feature request, I think it shouldn't be that difficult to implement with the help of GStreamer. I'll file it at Berlios with some workflow ideas.