Monday, June 30, 2008
Monday, June 23, 2008
In theory, GStreamer is everything a video editor has dreamed of... but I couldn't get hold of the developer guide (it's on development. How ironic). However, I have this hunch, telling me that I won't be able to access frames and audio samples directly, and instead I'll have to rely on plugins such as gnonlin.
Let's hope not. I mailed one of the Pitivi authors for guidance. Let's see what we find out. Cross your fingers.
Saturday, June 21, 2008
Example: In your project settings:
Under "Compiler settings", "other options":
[[if (PLATFORM == PLATFORM_MSW) print(_T("-mthreads -D__GNUWIN32__ -D__WXMSW__"));;]]
(Note: The double ;; at the end is a workaround a scripting bug)
As you can see, the one-line script is pretty much like C. the _T must have been incorporated for wxWidgets compatibility. In any case, Notice the compilation string: -mthreads, -D__GNUWIN32__ and -D__WXMSW__. The -D is a compiler define. I tried to add scripting to the #defines, but that didn't work, so I had to add them on the compiler command line.
The wx-config part is for Linux. I've found that experimentally, having this option in the project doesn't affect compilation under Windows.
The same can be done with the linker. Under Linker settings, other linker options:
[[if (PLATFORM == PLATFORM_MSW) print(_T("-mthreads -lwxmsw28u -lintl.dll"));;]]
Here I tell the linker to link two windows-only libraries: libwxmsw28u.a, and libintl.dll.a (libintl is used for internationalization). For posix environments (OS X or GNU/Linux), the backticked expressions are more than enough, but you could do the same with "if (PLATFORM != PLATFORM_MSW).
Under search directories, I add both the Windows and posix directories.
Compiler search dirs:
Linker search dirs:
Resource Compiler search dirs:
With this simple settings, you won't require having two different projects, one for GNU/Linux or Mac OS, and another for Windows.
This will help making cross-platform projects quite easy.
Friday, June 20, 2008
Once upon a time... the second most active developer, Nopalin, had problems with his harddrive and ended up installing Windows. The pandora box had just been opened on me. I had now to download MINGW, compile wxWidgets, download an additional libintl from gnuwin32 (it doesn't come by default with mingw), install subversion, and guess what.
As I had expected, I had compiling problems with the project, which was configured for a posix environment. I had to modify the global settings for Code::Blocks under Windows (something you must NOT do, that's what the project settings are for!) until I found out how to make the project build properties cross-platform (this is a capability not implemented in Code::Blocks yet, maybe they have scripting capabilities now, but I don't know of them).
And then wxDateTime method FormatISODate() outputs garbage on Windows. When that was fixed, I noticed some other problems: The buttons in the welcome dialog were invisible, and the new project dialog has the wrong size! :(
How did all this happen? Well, I'll be able to find out as soon as I make the project cross-platform. In the worst cases, I'll have to use the same tactic the C::B devs did: Use parallel project files. Ugh. Let's hope I don't have to resort to that. Sigh...
The problem: The thing's written in [obscure programming language which is neither Python nor C++]. Oh, you didn't know there was a programming language called [obscure], right? See, THAT's the problem.
I'll quote some text from the other programmer (I'll rephrase all paragraphs to protect the innocent from Google searches):
Basically I'm sick tired of having a non-working video editor and a lot of novel ideas that other editors lack. And I've rewritten it so many times that I've got to get this off my head.Later, I read:
All those four years I haven't been programming because of school. But this year might allow me to start working on [editor] again.
Lately I’ve realized that programming takes too much time from my life.... I almost made it work... I’m not a programmer. Programming in [obscure programming language] is my hobby, not my profession. I don’t get any money from it. It's a problem when programming takes about 80% of my time and I don't have time left for normal activities ... my projects will be put on hiatus. Feel free to ask me for rights to commit to the subversion of [editor]. You can take over the project for now. I hope I’ll get back to programming [editor] with a little more peace of mind after next year [written on December 2007].
And this, ladies and gentlemen, is why the Lone Ranger approach to Open Source programming WILL NEVER WORK. Do you have the slightest idea why so many Sourceforge projects are abandoned? It's like the Open Source Project cemetery. Well here's the reason: Those projects had NO programming teams.
I absolutely refuse to work on a hobby project if I have to do it myself, with tools I have to write myself because there are non-existing tools for an esoteric programming language (let alone a binding to [ famous multimedia library ] ).
Perhaps you'll understand now why I chose to use C++ and wxWidgets. There are at least 10 C++ programmers for every [obscure programming language] programmer. And there are at least 10 Windows users for every Linux user. If I'm going to have a programming team and a live project, I better use things where I can recruit most volunteers.
After all, a Non-Linear Video Editor is not something you can make in your garage.
Thursday, June 19, 2008
In other news, I opened a private mailing list for the developers so we can organize ourselves better. I'm also working on the developers guide so anyone can install the software required to compile and run Saya.
Then we'll start working, and hopefully in a couple (ok, maybe 4) weeks we will release version 0.1, code name: Aikuchi. That will mark the end of the planning phase and we will officially become "pre-alpha". Keep in touch! :)
Wednesday, June 18, 2008
An AVClip has a vector of effects.
An Effect consists of a map (string, FXParamTimelines). The string denotes the parameter's name.
An FXParamTimeline is a map of (integer, string parameter), where the integer represents a point in time.
(Notice that there are no bezier curves in this diagram. But there will be, don't worry about that).
Time is measured in milliseconds, but I'm going to replace the integer with a 64-bit integer so I can represent nanoseconds (this is necessary to be able to move audio clips around with one-sample precision in time).
One thing that I forgot to mention is that none of the data structures I present here, have pointers (pointers are EVIL!). This is so we can duplicate the instances of effects and clips easily. Since the indirection is handled through array (or vector / map) indexes, we won't have segmentation faults due to a badly dereferenced pointer. This also makes serializing and deserializing the data a piece of cake.
When copying a series of clips to the clipboard, I will replace the clip id's with new ones. Simpler than a Jedi mind trick. "These aren't the id#s you're looking for." *Waves hand* No pointer mangling, no headaches.
So this is the reason why I'm trying to focus on the User Interface first. Because it's the task which takes most of the time. And since I already got rid of the wxWidgets-specific classes in this framework, we can choose whatever UI library we want. But I'm sticking with wxWidgets for the user interface.
Here's a simplified UML Class diagram for Vidproject and associated classes:
The Video Project has a list of the currently used Resources (video and audio clips - i.e. files ). Additionally, it contains an AVTimeline which contains various sequences. Each sequence contains various audio and video tracks. And each track contains... surprise, a clip. The clip has an index indicating which resource it operates on. And of course, a list of effects and a transition.
Notice that a video clipboard is nothing but a sequence. Convenient, isn't it?
As you can see from this diagram, the memory consumption for these classes is minimal, since they're nothing but sparse data structures.
Now, I don't really know how Cinelerra or other video editors work. But one thing's certain: It's much easier to model the classes after the structure of the data we're going to modify. And this data is a timeline.
I already have implemented saving undo/redos for these classes, and you can impose a memory limit on how many undos/redos you can store in memory. So there goes the high memory requirement. Poof! :)
I'll upload this diagram to the webpage when I can.
Tuesday, June 17, 2008
I added a nifty debug log to our wxApp object, so we can finally know what's happening behind the scenes. This is excellent for hard-to-debug cases. And now, back to business, there are tons of things to do:
- Fix the updating of recent project menus. I currently use a flag, it's better to use a counter to see whenever it's been updated.
- Implement the audio/video presets in the new project menu
- Implement project creation and saving with XML.
- Keep working on the framework to do some video streaming (Jonathan's beaten me to that already, I must keep up in the race!)
- Update the website, write the class diagrams, etc.
Monday, June 16, 2008
Sunday, June 15, 2008
After a few hours of rewriting code, I managed to almost replace all. However, I'm left with a few stubs (I only committed a zipfile with the changes, I don't want to break the build in SVN). Here's a list of the new functions and changes:
- static std::string ioCommon::GetPathname(std::string fullpath); // UNFINISHED
- static std::string ioCommon::GetFilename(std::string fullpath); // UNFINISHED
- static bool ioCommon::FileExists(std::string filename);
- static bool ioCommon::FileExists(const char* filename);
- class BufferedFile; // UNFINISHED
- class TempFile; // UNFINISHED
- const std::string syString::Format(const char* format, ... );
- const std::string syString::FormatBig(unsigned long bufsize, const char* format, ... );
- enum sayaEventType;
- enum sayaYesNoCancel;
- class sayaEvtHandler; // This will be used for handling the events coming from the backend.
- class sayaConfig; // A wxConfig wrapper, it's an abstract class.
- class sayaConfigProvider; // abstract class. UNFINISHED; needs an implementation (wrapping) done in wxWidgets.
- const wxString std2wx(const std::string& str);
- const wxString std2wx(const char* str);
- ALL wxString references in ProjectManager were replaced by std::string
- Since wxWidgets uses the _() macro, all literal strings will have to be surrounded by gettext().
Now I have to go to sleep. I'm dead.
Saturday, June 14, 2008
So I'll start replacing the wxString references everywhere except in Main.h and Main.cpp for std::string . This will also help me get rid of the _T() and macros. For internationalization, I guess I'll redefine the _() macro to some internal function, used only by the backend.
It also means I'll make an intermediate class to pass the messages (like presenting dialogs). I think I'm going to like this.
Friday, June 13, 2008
Update: Fixed. To solve the SVN access Issue, I only had to replace the svn+ssh login method by https. I had to do some directory copying, but it was easy.
Thursday, June 12, 2008
The first decision I took was to use the SFML Mutexes class. Unfortunately, I stumbled upon the problem: If I include this for the wrapper, what will happen when one of the plugins actually uses the library? Would I get some duplicate definition? Should I copy the code and rename it?
Forget it. Enter google. I found two great resources for using Mutexes: computing.llnl.gov (which included a copy of the pthreads manpage) and wikipedia. Lucky me, the wikipedia tutorial included a cross-platform version of implementing critical sections! And it's GFDL, alright!
Now let's see if I can find a good example of a cross-platform thread-safe sleep() function.
Update: Done. Now I can go back to work on thread-safety for the audio/video wrappers.
Wednesday, June 11, 2008
And they're right. But I'm not writing a backend. I already have one. See?
From the webpage:
OpenVIP is a free video-processing tool for Linux and Windows. It consists of two parts:
- OpenVIP core, which can be used for processing multimedia files from command line, or as a C++ library linked to other applications.
- OpenVIP editor, which provides a user-friendly GUI to the core and is based on the timeline concept - you place multimedia files on the timeline, apply filters, transitions, ...
- Supports AVI, DV, MPEG, MOV, MP3, WMA, and WMV formats (via the FFmpeg libraries) as well as sequences of bitmap files (via the ImageMagick library)
- A lot of nice plugins including colour transformations, geometric distortions, basic sound processing and transitions between two clips
- A simple interface for developing your own plugins in C++
All we're doing is designing a professional user interface around the OpenVIP Video Editor framework. Because the framework is done in C++, designing the frontend is much simpler than starting a video editor from scratch.
The only problem is that OpenVIP is released as GPLv2, while Saya is GPLv3. I've asked the main developer to release the project as GPLv3 so I can add the framework right away. Perhaps his mail got lost, I guess I'll ask again. (Edit: I just got a mail from Antonin Slavik, the OpenVIP project leader. OpenVIP is now GPLv3 or later with linking permission! Road's clear!)
Still... we can't commit the same mistake other developers have by tying themselves to one framework and later realizing it doesn't do what they need. This is why we're making a whole "interface layer" in Saya, so the video processing and playback frameworks can be inserted as plugins. Some things will need to be implemented on our own (or outright stolen from mature frameworks), like threads, mutexes. There's a 90% chance that I'll borrow SFML's system module to deal with this part.
The rest will be a lot easier to handle.
Tuesday, June 10, 2008
* Organizing the team, getting team members. Some members haven't been able to be contacted live - but we got an Adobe Premiere video editing expert / advisor (edit: And tentatively another advisor, who happens to be a Sony Vegas expert and User Interface Nazi ;-) ) , a beta tester, 3 developers and 1 possible developer more (if I manage to get him online). Additionally, an old friend from school who's a college graduate is offering to join the project - but not before he passes a wxWidgets exam I assigned him :)
* Researching on various fields. Nopalin has been able to do his first wxWidgets programs, and is ready to work on the UI. b3rx is in the process of documenting the OpenVIP classes and some classes that have already been designed. I've been researching the use of SFML and have corrected various misunderstandings in the wrapper classes.
* The pluggable multimedia framework has been started. Abstract class AudioOutputDevice has been completed. Abstract class VideoOutputDevice has been started (as a stub). Actual implementations using classes derived from these, remain to be written.
What remains to be done:
- Completing the New Project Dialog and creating new projects.
- Creating an A/V streaming class for rendering video and audio on-the-fly.
- Creating a plain vanilla video player using the said classes.
If everything goes well, this will be done within the next 4 weeks.
That sounds just like what we're looking for. However, there is a problem, and it's that the project is (relatively) too young compared to SDL. How stable and reliable it is?
I'm sure that choosing a multimedia library is a problem many multimedia programmers face. They have to base their entire code on choosing one or another library. This is why I decided to design abstract classes as wrappers, so we don't have to face that decision. I hope that with SMFL we can have a working multimedia player soon.
Monday, June 9, 2008
Thursday, June 5, 2008
"GStreamer solves a lot of problems on the GNOME desktop but it doesn’t solve the problem of video editing.
Gst is a playback framework, and for video editing you need editing framework. The later is not, as it’s commonly believed, just a superset of the former. The MLT framework is an interesting example of the “video editing” architecture."A later comment from a reader explains further:
"I would also join the voices in suggesting that you reconsider your decision with GStreamer. In the last six months GStreamer has become more and more suitable for applications such as Diva, and Gnonlin is a particularly useful component here."
And here I was thinking that I could use GStreamer as a basis for Saya. I think I can explain the concepts with a small diagram:
This is, gnonlin is a PLUGIN which goes with GStreamer (if it's not, someone correct me please!). The problem is that this plugin is a video-editing component built upon a low-level playback framework. This is:
OS / Hardware
and that's a no-no. It's got the logic ALL WRONG!
Instead, what I want is something like this (the higher, the more user interaction) :
| (commands, events)
High-level Editor framework (track / timeline /
effect stuff handling)
| (playback info, commands, events)
Renderer (effects / mixing)
| Low level
| Playback Framework
| (i.e. SDL)
v Video Hardware
In other words, The decoder/files make the MODEL. The Playback/VideoHardware are the PRESENTATION (the GUI is another separate layer of presentation, so they're tied - in a way), and the Editor framework is the CONTROLLER.
Ladies and gentlemen, this is nothing but Model-View-Controller 101.
Update (June 23, 2008) : It seems I was mistaken in my assumptions regarding GStreamer and PiTiVi. However, the MVC pattern is still a rule I'm going to follow. Stay tuned.
Wednesday, June 4, 2008
Alas, it seems he chose another path (he's going to use GStreamer and Python). Why, oh, why??? I really don't know if he'll succeed, but I asked him to join forces. Sigh, if only we had managed to get in touch earlier. Jonathan claims that C++ is a monster hard to debug. Well, not if you use the latest GDB and Code::Blocks. Programming C++ with Code::Blocks is a breeze ;-)
Luke er Jonathan... JOIN ME, and we shall rule the galaxy together! Well, more or less :P