.sony

Calendar Update Part II - After The Release

2011-06-10T11:14:00.000-07:00

Following up on my promise to detail some of my future plans. But first, a word about that nasty memory leak.

Memory leak in pycairo or pango/cairo.

I am 80% confident that there's a bug in the python bindings shipped on ubuntu 10.10. I developed a minimal example, (full source here), which just draws some text repeatedly in a window. The relevant bit of this example is the following function:

def draw_text(self, cr, text, x, y, width):
        pcr = pangocairo.CairoContext(cr)
        lyt = pcr.create_layout()
        lyt.set_text(text)
        lyt.set_width(pango.units_from_double(width))
        lyt.set_wrap(pango.WRAP_WORD_CHAR)
        cr.move_to(x, y)
        pcr.show_layout(lyt)
        return lyt.get_pixel_size()[1]

This little script will om nom nom your memory -- over 100 mb of memory after just a few minutes, and I think we can all agree that even if this it's not the most efficient example, its memory usage should be nearly constant. I haven't yet opened a bug, because I still don't know which component(s) contain the actual leak. At least feel better that it's probably not something I'm doing wrong, which gives me the confidence to move forward with the release.

To mitigate against the leak, I threw in an optimization that ensures only one pangocairo context and one text layout is created, per expose event, per widget. This seems like a reasonable thing to do in general, so I will probably leave this optimization in even after the bug is fixed. Unfortunately, all this does is slow down the leak. The leak is still there, which means you still wouldn't want to leave the calendar application running indefinitely.

In passing I'd like to say that the idea that a pango layout needs to be associated to a particular context bothers me a lot. I wonder if I can get away with re-using layouts with different pangocairo contexts.

A better grammar

The current parser for recurrence expressions could use some serious help. The parser for the existing grammar is in the "proof-of-concept" stage of development. It's intuitive for simple things, but you will notice that parenthesis are needed to get correct parsing of more complicated examples. This is due to unresolved ambiguities in the grammar.

Most of the ambiguity could likely be resolved with associativity and precedence declarations, but when you're developing a new notation it's difficult to know in advance what the intiutive associativity is. Basically I didn't bother trying to get it right at all, and rely on parenthesis as a crutch. I recognize that this neither intuitive nor natural. Even I am surprised at how wrong the unparenthesized parse can be.

I am also aware that this approach is horribly anglo-centric. You would have to write separate parsers for every language you wanted to support, and it's a good bet that language constructs don't always map onto each other so neatly.

I am also playing around with graphical approaches to the same problem, but to be perfectly honest they've all sucked so far. I might do a separate posting on one of them just to show you how god-awful it is.

A mouse-driven way to add recurrence exceptions.

When dragging a recurring event, the current behaviour is to shift the entire recurrence pattern. This is a deliberate choice on my part: it's a novel feature that I want to highlight it. Unfortunately, many times time what you actually want to do is shift just the occurrence you've selected, and occasionally you want to move all the events after the selected occurrence.

Note, you can do this now: however, you have to go through the text interface. Even I find this quite tedious. The solution most obvious to me is to introduce control and meta modifier keys, but I have found that other users do not share my enthusiasm for mouse modifiers. In usability parlance, we say that modifiers are less discoverable than more explicit features.

Another idea would be to add some tool bar buttons that silently add modify the recurrence rules of the selected event. This would be fairly easy to do, but I have a couple of misgivings. The first is that the toolbar is getting crowded already. The second is that it won't be obvious what those buttons do at first. Clicking on the "except this occurrence" button will not produce an immediately visible change, only modify the behavior of future interaction. This would be especially be confusing if you hit the button by accident.

A more radical idea would be to place special "handles" on the clip itself. That way dragging from a handle could have a different meaning from dragging the clip itself. I am leaning towards this approach, but it is not without drawbacks of its own. For example, the handles compete for space with the event text. They will necessarily be small, which could hurt usability in a tablet.

Integration with other calendars

Syncing with Google Calendar and / or EDS

importing from, and exporting to, .ics.

This work ought to be pretty straight-forward given how the code is organized, but I haven't really looked in detail at the APIs / File formats I'd be working with. If you're interested in helping out with that, you know where to find me ;)

Event alarms

This might not be necessary if I can use something like EDS as the default back-end. But we'll see. Depending on how hard it is to map the concepts in my application onto the APIs exposed by EDS, it might be easier to roll my own implementation.

Calendar Update

2011-06-08T22:58:00.000-07:00

What a crazy couple of months it's been. I have been back in San Francisco since late February, but not found the motivation or time to post until now. Among other things, I've been to racing school, and I've taken up drumming as a new hobby. I also attended the meego conference here, where I was showing off my QML GES demo (another post about this is still in progress).

I thought I'd take some time to write about the work I've done on my desktop calendar application over the past several months. This will stop me from pestering my friends about it, which will make them a lot happier.

Major cosmetic face-lift.
Added an infinite-scrolling month view.
Added support for all-day events, which are displayed in a pane along the top row.
Added support for recurring events via a natural-ish language parser which supports english-like recurrence expressions such as "every two days from now until october", or "the 3rd wednesday of every month except this wednesday"
You can easily shift the entire recurrence pattern by dragging a single occurrence with the mouse
Possible to select the same block of time across multiple days.
Added support for editable event text.
Removed dependency on goocanvas. Since all the drawing code is custom, and the entire widget is re-painted on every expose, there really is no point in using goocanvas.

I am considering making an initial release, but would like to address the following issues:

I need to come up with a name. As my music teacher once told me, "Name your children." Anything is better than cal.py.
Adopt some system for installation, most likely distutils.
Currently the calendar widget leaks about 100kb of memory for every expose event. After a few minutes of scrolling and dragging it's pushing 100mb. It's obviously a bug, and I've narrowed it down to the functions I wrote to simplify drawing text. Disabling these functions reduces the memory consumption to a much more reasonable 15-ish mb, Either I am doing something memory inefficient in python, or I've found a leak in pango / cairo. I would very much like to understand that better.
I recently added all-day events. You can create an event as an all-day event, or you can drag an existing event to the all-day area. But you can't drag an all-day event back into the timed area.

I did some "usability testing" on a hapless friend of mine, and the results were encouraging but showed there was a great deal of room for improvement. Unfortunately, my select-then-add paradigm of creating new events is confusing for those used to applications like google calendar and evolution. In evolution I find the click-to-create behaviour frustrating. Google calendar, on the other hand, seems to get it right.

After the release
This post is getting long enough already. I've also got some ideas for subsequent improvements, which I will summarize here. TTFN

A better grammar
A mouse-driven way to add recurrence exceptions.
Integration with other calendars
Event alarms

Testing Interactive Code

2010-11-26T17:35:00.000-08:00

The preceding work on the Render and Project Settings dialogs in PiTiVi has led to some fairly complicated UI logic, and we haven't really got a good way to test it.

Imagine that we want to test the following trivial python UI

import gtk
import gobject
import random

def b_clicked_cb(button):
    clicked = True
    button.props.label = "Ouch!"

w = gtk.Window()
v = gtk.VBox()
b = gtk.Button("Click me")
b.connect("clicked", b_clicked_cb)
v.pack_start(b)
w.add(v)
w.show_all()

gtk.main()

We want to check that the button label is correctly changed after being clicked. Because of control inversion, we obviously can't code in a direct, sequential style. At the very least we must install a timeout or idle function into the main loop.

This is a naive approach:

def test_case():
    b.activate()
    assert b.props.label == "Ouch!"
    return False

gobject.timeout_add(1000, test_case)
gtk.main()

It doesn't work because of timing issues. The button's label won't change immedately after activate() is called, so you get the following error -- despite the fact that the the label clearly changed.

Traceback (most recent call last):
  File "figure_1.py", line 19, in test_case
    assert b.props.label == "Ouch!"
AssertionError

The obvious solution is to split the callback in twain:

def test_case():
    b.activate()
    gobject.timeout_add(1000, finish_test)
    return False

def finish_test():
    assert b.props.label == "Ouch!"
    gtk.main_quit()
    return False

gobject.timeout_add(1000, test_case)
gtk.main()

It's good enough for this trivial example, but it should be clear that as the complexity of the UI increases you'll end up with yet another maze of callbacks.

Last night I experimented with applying python generator functions to this problem. The idea is to (ab)use the yield keyword as a way of passing control between the test and the mainloop. It's similar to concepts presented in this paper, which describes how continuations can be used to solve the problem of control inversion in web programming.

Quick summary of python generator functions: the yield keyword in python is a limited form of either lazy evaluation or continuations, depending on your point of view. Usually we think of generators as sequences. For example this generator can be thought of as the sequence of positive integers from 1 to 10

def ints():
   i = 1
   while i <= 10:
      yield i
      i += 1

I can treat it as sequence and take its sum, or iterate over it:

print sum(ints())
squares = [x ** 2 for x in ints()]

What actually happens when squares is called is that an iterator is created and returned to the caller. Yield indicates to the interpreter that squares() is a generator, and that the state should be saved so it can be re-entered later.The fact that the state of the function is saved inside iterator allows us to think of the iterator as a continuation (generators are not as powerful as continuations). With this in mind, we re-write our test function:

def test_case():
    b.activate()
    yield
    assert b.props.label == "Ouch!"

Here the yield keyword is going to provide the same control flow boundary that splitting our code into separate functions did earlier.Now all we need is a bit of code to consume values from this iterator until it is exhausted. We do this using timeouts -- as we did in earlier examples -- which allows the mainloop to continue running in between the two halves of our test case.

def run_test_case(iterator):
    print "Tick"
    try:
        iterator.next()
    except StopIteration:
        print "Test Case Finished Successfully"
        gtk.main_quit()
        return False
    except Exception, e:
        print "An error occured"
        gtk.main_quit()
        return False
    return True

gobject.timeout_add(1000, run_test_case, test_case())
gtk.main()

This is already an improvement, but not yet flexible enough. Suppose we add a new widget:

def c_clicked_cb(button):
    def set_label_async():
        c.props.label = "Ouch!"

    gobject.timeout_add(random.randint(500, 5000),
        set_label_async)
c = gtk.Button("Async Operation")
c.connect("clicked", c_clicked_cb)
c.show()
v.pack_start(c)

Now we have a problem: we have no idea when the action triggered by clicking the second button will complete. Simply waiting for one second will not always work. At the very least we should be able to override the default sleep value. But it would be better still if we could wait until the label value itself is changed. That way if the action takes only a short time, we don't have to wait, while if the action takes longer than expected, the test can still finish successfully.In other words, we shouldn't just assume that after each step in the test we straight on to the next one. Let's factor out portion of the loop that does the scheduling:

class Sleep(object):

    def __init__(self, timeout=1000):
        self.timeout = timeout

    def schedule(self, iterator):
        gobject.timeout_add(self.timeout, run_test_case, iterator)

def run_test_case(iterator):
    print "Tick"
    try:
        scheduler = iterator.next()

    except StopIteration:
        print "Test Case Finished Successfully"
        gtk.main_quit()
        return False

    except Exception, e:
        print "An error occured"
        gtk.main_quit()
        return False

    scheduler.schedule(iterator)
    return False

Now we can easily customize the timeout for the second button specifying a Sleep scheduler with a different timeout.

def test_case():
    b.activate()
    yield Sleep()
    assert b.props.label == "Ouch!"

    c.activate()
    yield Sleep(6000)
    assert c.props.label == "Ouch!"

Actually we can go one better. We don't have to rely on timeouts for scheduling at all. For example, we can easily define a scheduler that will wait for a signal to fire:

class WaitForSignal(object):

    def __init__(self, obj, signame):
        self.obj = obj
        self.signame = signame
        self.iterator = None
        self.sigid = None

    def schedule(self, iterator):
        self.sigid = self.obj.connect(self.signame, self._handler)
        self.iterator = iterator

    def _handler(self, *args):
        run_test_case(self.iterator)
        self.obj.disconnect(self.sigid)

Adopting this is just a one line change to test_case():

def test_case():
    b.activate()
    yield Sleep()
    assert b.props.label == "Ouch!"

    c.activate()
    yield WaitForSignal(c, "notify::label")
    assert c.props.label == "Ouch!"

And we don't have to touch run_test_case() at all.I think this idea could be expanded into a framework for testing event-driven code. True, I would want much better error reporting. But with just that, it would be pretty straight-forward to cover every part of PiTiVi's interface except the timeline (for the Timeline, I need is the ability to synthesize raw input).If necessary, I can include other types of scheduling scenarios, such as waiting for a file or socket access. And, because I can work with the widgets directly, it's possible to verify conditions that would be impossible to check for under Dogtail or LDTP (which are both limited to what AT-SPI exposes, and run the test from a separate process).Full Source

And the winner is...

2010-11-26T14:31:00.000-08:00

The overwhelming majority went for version B. I changed the layout a bit, and a cleaned up version is what's in my branch now. Thanks, everyone.

Round 4

2010-11-18T09:36:00.000-08:00

So it's down to two competing ideas. At this point my brain is shot and I can't really decide between them. So I've decided to put them up here and see what people think. The best way to compare the two is right-click the images into separate tabs and switch between them a few times. Take a good hard look at each one.

What I'm looking for here is clarity of ideas, not cosmetic considerations like alignment and balance. That is, which of these dialogs is easier to understand?

Now that you've done that, does anything seem to be missing from either one that would make it an improvement over the other?

Round 3

2010-11-17T12:13:00.000-08:00

09:38 < bemasc> emdash: So, the true parameters here are the storage height,
storage width, PAR, and DAR.
09:39 < bemasc> that's 4 values in total, but only 3 degrees of freedom.
09:40 < bemasc> Personally, I would expose all 4, and update PAR after each change to DAR (and vice versa)

How about this, then?

Starting to look a bit better? I kept the notion of 'Display resolution' mainly as a source of feedback, but it occurs to me now that perhaps little rectangle that represents the display aspect ratio would be a better idea. Unfortunately that would mean increasing the size of the dialog again.

I'm actually not so sure that DAR and PAR should update each other, as I played around with it and it's somewhat frustrating. I think most of the time you're either targeting a specific format (for example, 480i) in which case you'd just choose a preset, or you'll be working with square pixels.

I think i need to add a 'constrain aspect ratio' check-button to make scaling a bit easier.

Project Settings Progress

2010-11-17T08:09:00.000-08:00

Re-worked it a bit this afternoon. But i'm still not happy with it. I think it might be too confusing to expose pixel-aspect-ratio to the user (hell, it's confusing enough for the programmer).

Another idea: present display width / height, optionally allowing the user to constrain to an aspect ratio. Separately present content (storage) width / height, and finally show the pixel-aspect-ratio as a read-only label, as a form of feedback. In other words, PAR would be computed working backward from display and content resolutions.

Mockup: The diagram on the bottom shows how updates
would propagate between widgets

PiTiVi Dialogs

2010-11-16T13:13:00.000-08:00

I am (finally) making progress on the two main dialogs in pitivi: "Project Settings" and "Render". I'll just get right to the point: Screenshots!

Render dialog showing video tab

Render dialog showing general tab

Project settings showing video tab

I spent most of my time today on project settings. I'm still not quite happy with it, as the current design is, well, huge. I think I could make it a little narrower and shorter. But that's more-or-less what I'm after.

The properties in the video tab are inter-dependent. One feature I wanted was for the widgets to ripple-update like a spreadsheet. To make this easier I wrote a class that manages a graph of widgets that you give it. Each widget can have an update function, and when the user changes one of the widgets, the the update functions are called starting from that widget in breadth-first order. During the traversal, updates are ignored to prevent infinite loops of signal emissions. This means I get my ripple updating behavior, but I only have to write a single, naive update function for each widget without worrying about keeping other widget values in sync.

Tomorrow I will implement presets, make sure the dialogs remember their state, and then test the hell out of them.

Side Project

2010-11-01T14:37:00.000-07:00

I started working in a calendar app. Right now you can think of it mainly as a prototype, but it's already got some interesting features. It's implemented in python, and while it relies on goocanvas at the moment, the calendar view is actually one big canvas item and could easily be ported to a stand-alone widget.

Interface

At first glance it looks like the week view used in most timelines. However, the view is an infinite scrolling area. There is no horizontal scroll bar, as this is a ranged widget. You move in time by clicking and dragging on the day heading. If you flick the heading, the calendar will continute to 'coast', slowing down gradually until it stops (or immediately if click on it).

I've found that I rarely look at the month view in a calendar application, so I sought to eliminate it. My calendars tend to have a lot of daily repeating events, making the month view seem cluttered. I mainly use it for skipping forward or backward to a specific week without loosing context, or to see a group of days that span a weekend.

With infinite scrolling I found I really don't need the month view. I don't loose context skipping around because of the smoothness of the animation. Moreover, it's no trouble to position the calendar so that Friday is in the middle of the window (the coloration on the heading serves as a reference). Expanding the window will reveal more days, not make the days larger. If you want to look two weeks ahead, you can -- even on an eee 901. I do plan to add a zoom feature, however.

Selections

Another feature I've added as a reaction against current calendars: When you click and drag on the calendar itself, it doesn't create a new event automatically. I've always found this to be annoying. When I make a selection, I am usually in some kind of tentative state of mind. Moreover, in thunderbird and evolution, it's possible to accidentally create events that are too small to be selected, forcing you to go into another view to delete them. Again, really annoying. I don't want the application taking some action based on my half-baked ideas. I much prefer it to wait until I'm sure of myself.

Undo / Redo

This applet is the first time I've implemented the Command Pattern in a program, and I'm curious to see how it pans out. The idea is that rather than employing low-level tricks to capture state for undo, you explicitly code the do and undo procedure for each action that your application supports. So far, New, delete, move, resize, and select can all be undone.

I think that selection is too often left out as an undo action. Doubly annoying because selections are ephemeral, disappearing the minute you focus on something else. Often I select things without thinking. I've put a lot of work into creating selections, only to have them evaporate due to an errant click.

On the implementation side, I distinguish between MenuCommands, from which menu and toolbar items can be created automatically, and MouseCommands (invoked repeatedly with new mouse coordinates before being committed to the UndoStack in their final state). Both types support a class-methods indicating whether they are currently available. This allows toolbar state to be managed, and for the controller to distinguish between one of several commands it might support.

One thing I found is that with this approach, the Controller becomes a general-purpose object which mostly creates and commits commands to an UndoStack in response to low-level events. The controller implements the logic for detecting drag, click, double-click detection. The commands implement more concrete actions, such as moving an event in the calendar based on given mouse coordinates. The Controller only needs to be specialized to add new input events. For example, flicking the calendar is currently implemented with a custom controller.

In the future I plan to make navigation undoable. I'm debating whether navigation commands should be posted to the main UndoStack, or into a separate stack that would be exposed as "Forward / Back". I'll have to try and see.

Development Status

If I follow this out to its conclusion, it would need at least the following features before I'd call it "Done".

Ability to edit the names of events

tricky because the whole thing is done in raw cairo

Some system for handling repeating events
convenience navigation buttons, such as "Today", "Last {Week, Month, Year}", "Next {Week, Month, Year}" etc.
Alarm notification system, and ability to handle alarms
Integration with google calendar (probably using libgdata)
some cosmetic improvements

The implementation of the alarm notification system worries me most. I have only a vague idea of how I would implement that, and it would need to be absolutely reliable before I'd consider using this program myself. I do have some interesting ideas for how to create alarms and repeating events in the UI, however.

Shopping

2010-10-30T14:51:00.000-07:00

Coming home Saturday night I was delighted to see that "The Co-Operative" grocery store nearest my house was still open. I went in to buy some things for next morning's breakfast.

There were two choices for bacon: one claiming Britain as its origin, and another from "the family farms of the Netherlands". At this point I must confess that my first reaction was "Eek! Why would I want Dutch bacon? I'm in England! The British bacon is for me." It seems I have begun to adopt a British point of view after only a few short weeks. That is until I looked more closely at the package and read that it consisted of "87% pork."

I was perplexed. How in he world can 13% of a slice of bacon consist of something other than pork? Some minutes passed, during which time I surveyed the contents of the package. Perhaps closer inspection would reveal a portion of the of the slice where some miscreant had inserted turkey or beef. At least that would be a reasonable explanation. Alas, I found no signs of obvious adulteration.

Unsettled, I chose the Dutch variety.

Scene from a small boat punting up the Cam (the river in Cambridge).

Some More Pics From Spain, Continued...

2010-10-09T13:51:00.000-07:00

I ate at this really neat cafe in Figueres.

The bathroom was entirely stainless steel, and used car door handles.

Here I am, trying to be artistic

There's a lot of really good street art in Figueres...

...as well as the more typical variety.

More scenes from Figueres.

Waiting for the train back to Barcelona.

In front of the Dali museum, with a dopey grin.

It's worth the trip. Thanks for the tip, Kat!

A closeup of the the dots on the exterior of the

building. Each one appears to be slightly

different.

Some More Pics From Spain

2010-10-09T13:38:00.000-07:00

I've been out of Spain for a couple of months, but I still have plenty of photos I never posted. Truth be told I found it a bit painful to manually upload them. I don't know why it never occurred to me to use Picasa until just now. Enjoy.

Tarragona

Scenes from a public concert in Montjuic

A warm waffle, covered in chocolate sauce and sweetened condensed milk. Purchased from the above stand.

2010-10-04T13:43:00.000-07:00

So, I'm in Cambridge, UK now. I've been meaning to post up for days but haven't quite found the time. I'm going to try to get some longer posts out in the next couple of weeks, but for now I'll settle for something short and simple.

Quote of the Week

The other day I was walking through Midsummer Common, which is a big green field near the center of town that normally contains quite a few cows. Instead there was some kind of bicycle race event and the place was packed with people and cycles. I smelled the smell of good, greasy food -- like the kind you get at a county fair or a race track -- so I sauntered on in. As I was munching on a tasty roast pork sandwich (with stuffing and apple sauce) I overheard the following snippet of conversation:

Boy (in a whiny English kid voice): So, can I mum, can I?
Parent: No Roger, you've been an absolute muppet today!

That's the first time I have ever heard the word "muppet" used this way. I wasn't sure whether it was a fluke or not, but while house-hunting today, my would-be roomate used the term in the exact same way to describe someone who forgets to lock up the house when they're the last to leave.

Nerd Section

Some time ago I got distracted by a silly idea that just wouldn't leave me alone. It kept gnawing at me to follow it up, so eventually I gave in and started hacking on it. Have a look at the following code. I'll give you three guesses to tell me what language it's written in.

     DEF (id(outer) FUNCTION(id(x)
       FUNCTION(id(ignored) VAL(x))))

     DEF (id(one_ret) APPLY(VAL(outer) _(1)))
     DEF (id(two_ret) APPLY(VAL(outer) _(2)))

     PRINT (APPLY (VAL(one_ret) _()))
     PRINT (APPLY (VAL(two_ret) _()))

DEF (id(count_backward) FUNCTION(id(x)

           IF(NOT(VAL(x))
              PRINT (_(done))
              BLOCK(
                PRINT (VAL(x))
                APPLY(VAL(count_backward) MINUS(VAL(x) I(1))
)))))

No, it's not LISP. The parenthesis go on the outside of the expression.

No, it's not python. Notice how there's no commas separating arguments.

Did I hear someone say haskell? Now you're just being silly.

It's actually C. Through abuse of variadic functions and preprocessor macros, I implemented a functional, dynamic language directly in C. For the curious, you can see the full example at my git repository, http://github.com/emdash/goof/blob/no_commas/simple.c:

I did it for fun, but it does work. Bonus points for explaining how it works in a comment.

Returning Home

2010-08-16T03:59:00.000-07:00

I'm a bad blogger, I know. I didn't even post before leaving spain :( In the last couple of weeks I managed to do some traveling around Catalonia, seeing Taragona and Figueres. Dali Museum is really quite awesome in Figueres. Taragona has roman ruins and a good beach. Have pictures, but not on this computer. Will try and post them soon.

I'm currently in Boston visiting some friends on my way back home. My friends live between Harvard and MIT. Walked around the campus and then my friend took me down town to see boston common, and the harbor. We also passed by the grave yard where Paul Revere, John Hancock, and the victims of the Boston Massacre are buried. John Hancock's tomb stone is the largest of the bunch. While walking through downtown Boston we ran into Richard Stallman. That's all I've got for the moment.

Today I am thinking about busing to New York city and back for a bagel, a hot dog, and a pizza. But we'll see. Jet lag and my friend's cat have combined to give me a bit of a jumpstart on the day. I'm looking at the schedules right now.

Getting Ready for GUADEC

2010-07-23T15:20:00.000-07:00

The last few days have been a welcome reprieve from the oppressive summer heat. Thank god for that. Of course it cools down just before my departure to the Netherlands for GUADEC.

Going to GUADEC next week. There's going to be a big PiTiVi hackfest involving the majority of the current PiTiVi contributors. I'm really looking forward to some serious progress.

Also hoping to give a lightning talk about some hair-brained ideas I have for UI development.
Last week, my friend Aaron came to visit and we rented scooters and rode around the city. It was hot, but fun. Enjoy the photos.

This is us up at Parc Guell.

Did you ever think you'd see Aaron on a scooter?

Ear Worms

2010-07-22T06:02:00.000-07:00

Enjoy the rest of your day.

Moto Mania (part 2)

2010-07-18T10:22:00.000-07:00

Let's kick this off with the obligatory motorcycle-road-trip-poster-shot-of-the-bike-on-the-side-of-the-road picture.

Oh GOD! Helmet hair! Make it stop!!! Why? WHY?!?!

Andorra is a beautiful country, and it's impossible to do it justice in photographs. All I can do is give you an impression. I rode into Andorra la Vella, the capital city of Andorra. It's a pretty compact city, having been built within a steep mountain gorge.

The downtown area contains quite a few shops. As it was sunday, most of them were closed. At first I thought that I was impervious to shopping districts. After all, I have all the clothes I need, my computers are up-to-date, etc...

Uh oh, I hadn't counted on this! Run! Run for your life!

These are the largest ball bearings I have ever seen. (Note: the author is aware of the comedic possibilities presented by the preceding statement and encourages you to submit your witty retorts as comments. For example, "that's what she said")

On the way out of town I discovered the sweetest back-road ever. It was nothing but endless switchbacks climbing higher and higher into the mountains. Had I a week to travel I'd have followed this road to wherever it led.

Just across the Spanish border is the city of Urgell. Above the town is a castle, which has been recently converted to a hotel. I think Americans suffer from castle-envy. To us, of course, anything built before the 1950s is considered old enough to be considered an historic land mark. Some media mogul builds a 'castle' on the coast of California and suddenly you have a major tourist attraction. I couldn't help but feel a little bit jealous noticing the multitude of ancient forts and castles dotting the Spanish country-side. As the time I had spent on the road was getting dangerously close to two hours, I finally decided to poke around one.

Here I am all squinty-eyed trying to take a self-portrait with the castle in the background.

I'm sorry guys, this just isn't as easy as it looks.

I did my best to imagine pouring boiling oil on hoards of invaders from this wall.

I took my sweet time coming back to Barcelona through the country-side. At one point I saw signs that read "Bar" and decided to follow them. When I reached the end of the road I discovered that "Bar" was, in fact, the name of a very small town and realized that I had been sadly mis-lead. But seriously, it's just freaking gorgeous out there. Coming down the hill from here I had to pause to wait for a shepherd to move his flock across the road. I really should have got a picture of that :(

I passed by Montserrat on the way back, and decided to take a little detour. The road over the mountain was so good I decided to ride it twice! Also, Throughout my whole trip I had noticed graffiti like this, and finally had an opportunity to take get a picture of it.

Whoo! Self-timer! But...hmm... something's not quite right.

There, I fixed it.

Brrrrrrrrrrr brrrrrrrrrrrr brrrrrrrrrrrrrrrrrrrr brrrrrrrrrrrrrrrrrrr

Home sweet home. Time for a beer and a kebab!

Moto Mania (part 1)

2010-07-18T08:32:00.000-07:00

As you probably know, I am mental about motorized, two-wheeled things. I used to own two motorcycles back home, but sold them both before coming here. It didn't take long for the withdrawal to set in, and after a couple of weeks I had made arrangements to rent one for the last weekend in July. I settled on Andorra as my destination, an independent country on the border between France and Spain.

At this time I would like to take the opportunity to share a few of observations about driving in Spain:

It isn't as difficult as I have been lead to believe.
Roundabouts are awesome!
The Spanish seem to have at least as much contempt for speed limits as Californians.

This is not to say that driving in Spain just like driving at home -- Far from it -- But adaptation comes naturally if you only pay attention to what other people are doing. The reality is that the rules of the road that matter are the unwritten ones. The rules that govern the flow of vehicular traffic are as cultural and as those that govern fashion and diet. I think that the American perception that European drivers are particularly crazy arises from the failure to understand this. Other drivers will do things that seem unexpected with astonishing frequency; however, this is simply because (as a foreigner) you do not understand the normal flow of traffic. It is you, the foreigner, who are the unpredictable, crazy one.

Roundabouts are really quite marevlous because they allow you to choose between any of the available directions (there will be more than four) in an intersection -- including going back the way you came! And not only that, but you do not have to make your choice in a split-second. You can remain in the roundabout for as long as it takes to become confident in your decision. This is most appreciated by people, such as yours truly, who are never sure of:

Where the are,
where they've been,
or where they are going.

(The exception to this is that within the city of Barcelona there are stoplights in the in the roundabouts themselves, so it can take quite a few minutes to complete a 180 degree arc).

According to wiki-travel:

Between cities, drivers are required to have some rest every 2 hours they drive--there's a fine if you don't follow. It's unclear how it's enforced, however.

This might seem like a potential inconvenience, but it's actually quite useful for male motorists wishing to retain some shred of dignity: you are not lost, merely looking for a suitable place to take your mandatory two-hour break. I would like to assure the reader that during the entire weekend I remained within full compliance of this statute ;-)

In the city it's pretty much impossible to get going very fast because of the congestion. In the country-side on the other hand, where the speed limit is typically a mere 90 kph, there was a 20-something driving a 118d hatch-back eager to go sailing past me at 140 on every long straight.

Look at this seedy fellow trying to rent a motorcycle. Can you believe they just let people like that ride those things in public? I don't trust him one bit.

I set off with a bit of a hang-over, having imprudently gone to the Belchica bar with friends to watch football the night before. I got a late start, and was not exactly in the best of spirits.

The desert a few miles north-west of Barcelona, taken while, *ahem*, looking for a suitable place for my mandatory rest period. Out in the desert in full leathers, it was (predictably) very hot. I decided, therefore, to take a few extra breaks.

After a period of time not exceeding two hours, I pulled into a sleepy town wishing for some lunch and a bit of a nap myself....

...and the idea of cold tomato soup had never seemed more appealing.

Octane in Europe seems to be a bit higher than at home, though perhaps they use a different scale.

By chance I had stopped to fill up at a rather auspicious petrol station. I could not help but go inside. The museum contained quite a few rare examples of classic motorcycles, but I won't bore you all by rambling on about the history and evolution of two-wheeled transportation in cryptic, technical jargon

...much. For example, this horizontally-opposed twin (above) has a completely-exposed over-head valve train. Note the lack of foot shifter. Below is a motorcycle powered not by a jet turbine, as it might appear, but a wankel rotary engine. It is described as being a rare and "largely untested" configuration.

Some things never change.

As I approached the Pyrenees, the weather became much less hot and much more moist. So moist, in fact, that I'm pretty sure it was raining. I arrived in Andorra at last, soggy and tired.

Andorra, it is claimed, is the only country in the world with two heads of state: one being the president of of France, and the other is the Spanish bishop of Urguell. So, after checking into the Hotel Coprinceps ("co-prince hotel"), I went for a bit of walk to take in my surroundings.

I look unhappy here because the US had just dropped out of the world cup :P Let's try this again...

I hate to end on a picture like that, so let's all take a look at the view from my hotel room. (To be continued..)

2010-07-11T15:12:00.001-07:00

Well, Spain's won the World Cup final. The fireworks here are pretty loud, but at least I don't live down town. I think I'm going to go wander around and bask in the chaos.

2010-07-07T14:31:00.001-07:00

...

Beep beep

Beep beep beep

Beep beep beep beep

BEEP BEEP!

¡España!

Beep beep

¡España!

Beep beep

Beep beep beep

Beep beep beep beep

BEEP BEEP!

¡España!

Beep beep beep

Beep beep beep beep

BEEP BEEP!

Beep beep beep

Beep beep beep beep

BEEP BEEP!

Beep beep beep

Beep beep beep beep

BEEP BEEP!

...

2010-07-06T13:42:00.000-07:00

Without the rigor of mathematics all science would collapse, but not all inhabitants of a building need to know all the spars and girders that keep it upright. Factoring off certain detailed knowledge to specialists reduces the intellectual complexity of a task, which is one of the things computer science is about. -- Parsing Techinques: A practical Guide (http://www.cs.vu.nl/~dick/PTAPG.html)

San Joan

2010-07-01T14:03:00.000-07:00

San Joan is a major holiday in Barcelona. Actually it's the eve before where the major partying happens. So far as I could tell, it's a good excuse get together with friends and have a barbeque, then make lots of noise with incendiary devices.

I heard you like meat, so I put some meat in your meat so you can eat meat while you eat met. Seriously, that is one massive steak.

Life is like a bowl of cherries: hmmm... wait, how does that go? Come to think of it, why do I have a picture of a bowl of cherries??

No, those aren't Candies, they're explosives :)

Hmm. Maybe it's time to lay off the pastis.

Un día en la vida

2010-06-28T15:02:00.000-07:00

My Room

The view from the balcony

Un café cortado. Mmmm cafééééééééééééééé.

El metro viene, el metro se va...

Purple?

There's art everywhere you look...

...but at times it's all a bit surreal.

Previously, on Star Trek: The Next Generation...

A little "self-timer" action...

Vanessa and Hubert, these are for you!

The sun sets on the plaza....

...The sun sets on the cathedral

Tonight, pintxos for dinner!

Oops!

Almost Home.

Goodnight everyone.

¡hola, mundo!

2010-06-21T14:20:00.000-07:00

I write to you from the from the wonderful city of Barcelona. Still hacking on PiTiVi occasionally, but presently spending most of the day on "not-pitivi.".

I've been hacking on the render dialog for a month or so, and it's become a real monster of a branch. But, hopefully, it'll really clean up that part of the application.

It's been exactly a month since I uprooted myself from San Francisco, and it's been quite an adventure. Unfortunately, I spent too long fiddling with my cell phone and camera and it's bed time now. I'm too tired to summarize everything that's happened, so I'll just tease you with a few more photographs.

Rest assured, I'll come back soon with "accounts" of "events" and stuff.

I miss everyone back home! Goodnight.

2009-04-25T11:13:00.000-07:00

To the three undo strategies I mentioned yesterday, we can add a fourth: widget-level undo. Unlike the others, this one works entirely on the UI side. The basic idea is that since all UI flows through widets, all you need to do is keep track of when those widgets change. An implementation of this idea is provided by the gtkundo library.

This seems like a good appraoch for input dialogs. You can reverse even small user actions that never actually propagate changes back to the model with relatively little effort (since a library like gtkundo provides all the code necessary to handle stock widgets), but it won't work for the timeline canvas itself, since this is a custom UI element. In any case I don't see python bindings for this library, so it would be difficult for us to use it anyways.

The concept itself is useful, and I plan to study the code (which looks pretty simple).

Okay, take a deep breath...

2009-04-24T21:51:00.000-07:00

I'll try and keep this short and sweet. Long story short, I'm still kindof obsessing over how to implement undo/redo in PiTiVi. The reason for this is pretty simple. Now that we have file support again, it's the one thing that keeps pitivi from feeling like a real application.

I decided to do a tiny bit of research on the subject (in other words, type things into google), and the main thing I learned is that there's no single answer to the problem, and every application ends up solving this in their own way. There's basically three main approaches to doing undo:

Brute Force (push a copy fo the entire document every time user makes an action)
State Capture (keep track of just the changes to the model at a low-to-intermediate level)
Command Pattern (basically what I presented in my last post, keep track of high-level user actions)

Then there's some largely-irrelevant, "magical" approaches, such as using low-level operating system features to directly and transparently keep track of when memory is read from / written to which won't really work for us because we're using Python and not C++, and because we actually want our code to work on more than one operating system.

In almost all cases, Undo support is very tightly coupled to the model side of the application, which is interesting because I had originally thought of it as more of a User Interface type of feature.

Brute force is pretty self explanatory. After every user action, a deep copy of the entire document is pushed onto a stack. Undo is basically just replacing the current document with one of these copies.

State capture is conceptually similar to the brute force approach, but more memory efficient in that only changes to a document are recorded. Depending on the architecture, it may be more or less work to implement a state-capture scheme. Inkscape, for example, has a dual-layer model. The top layer is domain-specific stuff, what they term the "svg" layer. The lower layer is a type-agnostic tree, what they term the XML layer. At the lower level, everything is just elements and attributes. Whenever this lower layer changes, the upper layer, and in turn the UI react (though not always the other way around). They use this lower layer to support undo/redo, and file input/output. Neat, but unfortunately PiTiVi isn't organized in quite this way. The nice thing about their approach is that it's clean: they can add, remove or change SVG-layer classes without affecting either file or undo functionality.

Most applications seem to orgnize themselves around the command pattern. This means that you essentially create an input language of "commands" which can be applied to "documents". Every user action is a command of some sort, including direct manipulation. The nice thing about this approach is that you can easily wrap it around an existing code base. Undo and Redo can then implemented by managing the history of commands applied to a document. With a little extra work, you can even allow for things like selective Undo/Redo, and use the same interface to support scripting. The down-side is that you have to manually specify every action and its inverse, and make sure that they work properly. This can, over time, become quite an onerous task. It's also not yet entirely clear how the command pattern fits with direct-manipulation (drag-and-drop type actions).

So, basically, I still don't know how I'm going to proceed, but at least I have some idea of what approaches have been tried in the past, and their associated trade-offs. I would favor some kind of state-capture-based approach, but this might require some labor-intensive refactoring the core classes. I think over the long run, the maintenance concerns associated with the command pattern would make this near-term refactoring more than worth it, and their may even be some cool python tricks (like using a meta-class) that would even minimize this maintenance effort. I guess the moral of the story is that this isn't strictly a UI problem, and so it's not something I'll be working on entirely on my own.

2009-04-10T20:17:00.000-07:00

Going back into a design phase now, so I thought I'd share what's on my mind. Some of this is similar to things I've talked about before, and other stuff is new. Mostly i'm just trying to clarify in my mind what I'm planning to do and how I'm going to do it.

Where we are now

basic editing works
text-based configuration file saves useful settings
limited support for changing priority
thumbnailing pretty mature (thumbnails are now cached, and there's a maximum limit to how many will be processed at one time)
seeking with keyboard added
better menu layout with more keyboard shortcuts
finally can have audio only files

Short-Term

lots of little bug-fixes
ripple / roll editing
slip-and-slide editing
don't let clips overlap each other (since we don't support transitions yet)
user preferences dialog (for the things we can already configure)
revamp export settings and render dialogs
clean up error reporting in the timeline
viewer will seek to appropriate places while clicking and dragging stuff in the timeline
project file support: has been integrated into the UI but back-end issues have kept us from merging

Medium-Term

property interface for timeline objects
revive the property editor
finally implement titles
ability to rotate and

Long-Term

keyframing (depends on property interface)
effects (depend on keyframing)
mixing and volume on video and audio tracks (a naive implementation wouldn't be hard, but as I understand it there would be performance problems for video, and dealing with those might take longer)
supporting the X clipboard (for copy/paste)
undo / redo support
secondary viewer, so we can do things like show two edit points simultaneously (not really sure how we'll support some of this in the back-end, since it essentially will essentially require having duplicate copy of the timeline)

Rethinking Commands and Selections

In the noun-verb UI, you first select what you want, then you issue a command to do something with it. Your data is represented as objects on a canvas, and selection specifies what you want to do. Commands are operations that manipulate or act on the selection. My ideal UI is one that has no tools at all. Everything you want to do you can do simply by defining a selection and issuing a command.

In a few, frequently-used cases, you can specify the noun and the verb at the same time. For example, performing click-and-drag a clip in the timeline, but really this is just a shortcut for "select this. now move it here".

PiTiVi's notion of selection needs to improve. Right now, we have only the most rudimentary notion of selection. Basically, you can only select entire TimelineObjects. Some commands, such as "ungroup", require only a single track-object as a parameter. Other commands, such as "move", need a position as a second parameter. Still others, such as "trim", might want a position and a duration.

To address this, I propose adding two new selection primitives: Regions and Markers. A region represents a slice of the timeline. It's just position and duration, with no content. A marker is just a special case of a region, with no duration. Here's a concept drawing of what timeline regions and markers might look like:

What I haven't settled on just yet is how you'll interact with them. I can't decide if they should be timeline objects in their own right, that you can directly manipulate (in which case there might be many regions you can select, drag around, resize, delete, etc), or if they should work more like the selection marquee in gimp (i.e. there's only one active region or marker at any time, which may or may not be contiguous).

However I decide to do it, there will be two main ways of creating regions (and markers)

directly, i.e. by clicking and dragging on the canvas
with the playhead

In the latter case, there will be a key, such as M, designated for the creation of regions. Pressing and holding M will create a new region with one end-point at the current playhead.

When the playhead moves -- either while seeking or during playback -- the area between the start of the region and the the playhead will hilight.

When you release the M key, the region will be completely defined. A threshold value will be used to determine whether or not a region or a mark should be created.

With this extra notion of regions and markers, it should now be possible to express most operations as imperative commands. For example, to split a clip:

you would select first the clip(s) and define marker(s)
invoke the split command either from a menu, toolbar, or keyboard accelerator. The split command would then split the selected clips where they intersect with a region or marker.

You could similarly define a define "trim" command

first select the desired clip(s) and define region(s)
invoke the trim command. The trim command will remove the portion(s) of the selected clip(s) that intersect (positive) or don't intersect (negative) the region(s)

Alternatives

I have already thought of one alternative. Instead of regions, make many more commands quasi-modes instead. Simple commands will still work with a single tap (or menu activation), but more complicated commands which can take a timestamp and/or duration as input would work like the M key described above: pressing the associated accelerator specifies one end-point, and releasing it specifies the other. If the playhead moves, the area in between the two end-points hilights. When the key is released, the command terminates with the final position of the playhead.

For example, trimming (start points) could work as follows:

move playhead to desired initial position
press and hold '['
start position of selected clips will snap as near to the playhead as possible
while seeking the playhead, the start positions will update
when the '[' key is released, the command is terminated at the final playhead position.

One benefit is that we don't introduce strange abstract primitives to the UI. This approach mirrors the click-and-drag structure of the existing mouse commands, which will probably make implementation easier (perhaps even sharing code). It can work with keyboard and mouse at the same time (hold '[' while scrubbing the ruler, for example). It's not modal, but it isn't noun-verb, either.

This approach might become unwieldy on the keyboard. The keyboard accelerators would have to be very carefully placed so as to make one-handed operation possible, and therefore become dependent on the current keyboard layout, requiring that we also have some way to configure the short-cuts. In addition, the shear number of keys that could become involved might make some actions (such as doing a ripple-edit while seeking with the keyboard) impossible to perform. And you might have problems with conflicting modifier keys (shift and ctrl are already used for keyboard seeking, so you can't use them for any command of this type without re-defining the keys used for keyboard seeking).

Implementation

Common to both approaches are the following: almost all user interaction is defined by the current selection, and the desired command. The roadmap for implementation involves the following

Selection

Selection is a class looking something like the following:

-contents (set of objects)
-history (stack of previous selections, uniquely identifiable)
-current playhead position
*"changed" signal
+setToObj()
+addObj()
+removeObj()
+setTo()
+clearSelection()
+getSelectedObjects([type,...])

The timeline selection methods can be moved into this class. With a few exceptions, UI will mainly work with the Selection class from now on.

we can now put in the boilerplate code to get the selection from the currently-focused widget.
For the TimelineCanvas widget in particular, we'll extract the current selection data from our private Selection object
now we can support the X clipboard
the selection object should be able to contain any core object we care to select
we need to be able to iterate over all the items in the selection, optionally filtering out the types of objects we don't want. For example, I could get a list of just the track objects included in the current selection. Or I could get a list of all the timeline objects which intersect the current selection (i.e. at least one of their track-objects is selected), or I could get a list of just the keyframe objects included in the selection.

Implement regions

...after I work out which approach to use

Commands

Commands are classes looking something like the following:

-name
-label
-description
-default accelerator
-selection
-stock_id
+do(selection_id)
+undo(selection_id)
+set_available(selection)

The idea is to refactor allmost all UI interaction in terms of Command objects. There are certain things we can do with them:

commands can be installed modularly
we need some global registry of all installed commands
automatically create menu items and toolbar shortcuts for all commands
refactor most user actions into Commands
when the selection changes, all of the installed commands peek at the selection and decide if they should be sensitive or not.
some minimum interface needs to be supported by all selectable objects, because some commands (such as delete) should be universal
but other commands will access specific instance methods, and they shouldn't be active if the current selection doesn't contain the right objects
we can easily do unit testing on the UI now, because now have a programatic way to invoke most actions

Undo / Redo

somewhere in the UI, maintain the undo and redo stacks.
All completed commands get pushed onto the undo stack when their do() method finishes successfully
undo pops the undo stack and calls undo() on the command
the command is pushed onto the redo() if it finishes successfully
for click-and-drag commands, we need to push equivalent commands onto the undo stack
need to record the state of the selection at each command invocation

Unresolved Issues

On which side of the core-ui split do commands, selections, and regions fall? what about the undo stack?
if selection seems to be both a core and a ui notion, does it make sense to have two Selection classes, a core Selection which just handles back-end objects, and a UI selection which wraps the core selection and also handles UI-specific details, like the X clip board
should commands be entirely static classes or do we need to instantiate them?
do we really need to maintain a separate selection history, or can we just push the current selection on the undo stack along with the appropriate command? maybe we make selection management internal to command objects (i.e. we set save and restore the selection as appropriate during calls to do() and undo())
would it be good or bad idea to consider the playhead the default marker?
would it be a good or bad idea to consider the entire timeline the default region? maybe this should be command-specific.

Waveforms \o/

2009-01-19T12:03:00.001-08:00

There you have it: fast, asynchronously-loaded waveforms for pitivi. It turns out it was useful to get thumbnails working first. Essentially the waveform is stored as a series of thumbails, and nearly the same drawing code is used to draw the waveform segment as to draw the thumbnails. The waveform segments are drawn into cairo surfaces directly from the sample data, which is fetched using a gstreamer segment seek.

Right now this branch is based off pitivi 0.11 so this could make it into the next minor release. Adding it to the current master branch will require a bit of refactoring. I might also be able to speed up rendering of waveforms a bit, which would mean that the previews will take less time to appear.

As for the cache, I haven't yet decided what scheme I want to use yet. I think LRU or MFU would be good enough, but it's possible that direct mapping would work as well and be much more efficient.

An interesting issue

I have been experiencing choppy audio within pitivi. What's interesting to note is that when you zoom in far enough, the audio waveform has gaps in it. And these visual gaps in the waveform seem to roughly co-incide with the auditory ones.

Thumbnails \o/

2008-12-28T20:55:00.000-08:00

After several weeks of often-interrupted effort, I have finally made progress on support for thumbnails on video clips. Thumbnails are fetched asynchronously and then cached. I have optimized the drawing routines to only draw the visible portions of each clip, so the thumbnail strips render quickly and do not seem to significantly slow down dragging or trimming on my machine. I'm fairly satisfied with the results. What remains now is to implement a caching strategy so that we don't eventually consume all available memory with 50x50 thumbnail images. I'm working on this in parallel with waveforms, which present slightly different challenges. Eventually I hope to modularize both thumbnail and waveform code in such a way that we can easily add new media types to PiTiVi, or optimize special cases such as still images (which just display the same thumbnail repeatedly).

In other news, I have finished my undergaduate education at UC Berkeley. Man, that feels good.

Mockups

2008-11-16T23:16:00.000-08:00

Recently I've realized that there are a number of things about PiTiVi that were and still are not clear in my mind. So, I've been working on improving the PiTiVi UI design document, particularly with mock-ups -- these are still images or sequences of still images which show how the program is supposed to behave. I've been working for quite a while, but the longer I work, the more unfinished things I seem to notice. On the other hand, it's a pretty amazing experience to watch previously vague ideas start to "gell" into something coherent. I actually have a much better idea how pitivi's features fit together, and where I still need to put more thought into the design.

You can see some of the work i've uploaded here: http://pitivi.org/wiki/UI_Design. At first it was slow-going. It had been several versions since I'd used GIMP. Also, it was a bit tedious to build up components like long strips of 50-pixel high thumbnails. Now that I have a library of bits and pieces, it's a little easier to build the scenarios I dream up. Anyways, I think I've identified most of the areas that I need to develop image sequences for.

As always, I welcome your feedback. Also, if anyone wants to help with this, talk to me.

bling

2008-11-08T17:26:00.001-08:00

I've been working on waveforms for audio clips. It's not quite usable yet: having these waveforms slows down re-drawing considerably, which brings the entire GUI to a screeching halt. At least I was able to get this screenshot, which looks like progress to me.

The Shape of Thigns To Come

2008-11-02T23:17:00.000-08:00

I decided that the default layout of PiTiVi's controls was pretty stupid, so I spent some time tweaking that today. First, I adjusted the way in which space is divided among the three main interface components. By default, you'll see this arrangement. When you resize or fullscreen PiTiVi's main window, only the timeline area grows, unless you explicitly move the separator. I think most users will appreciate this.

Today also marks the introduction of two new features: Detachable components, and the property editor.

Detachable Components

This feature you to separate each of the tabs above the timeline into a separate window by clicking the button next to the tab label. When you close the new window, the tab returns to its original position. Eventually the viewer will be detachable as well, but my first attempt at implementing this caused PiTiVi to crash immediately! Since we're planning to clean up the viewer code soon, I decided it made sense to hold off on making it detachable. Why did I spend time on this? Because some people will have multiple screens on which they'd like to work, and we want people to be able to full-screen the viewer if they choose. There's currently no way to full-screen a window across multiple screens. I also suspect that the default layout is going to feel cramped for some people, and I decided that the most flexible approach is simply to let people move whatever component they want into its own window.

Some of you may have noticed that I did things a little differently than other GTK+ applications. The truth is that I don't really like the drag-and-drop tab approach taken by the gimp. For one thing, it's not clear that the feature exits in the first place, since there's nothing to clue you into the fact that the tabs are movable (hell, I didn't even know you could do that...). This system has the advantage of simplicity. Either a component is in the main window or its own window. To detach it, you click the button next to its name. To restore it to its original location, just close the window you opened. PyGTK makes it really easy to pull this kind of trickery.

Property Editor

The Property Editor is going to become a central component of PiTiVi's interface. This is where you'll have access to the 'special' features of the objects you select in the timeline. As an example, consider the use case of still images. You may import a large image and wish to crop it down to a section. The image may be a portrait and you might want to rotate it 90 degrees. At the moment, we have code to support images, but up until now there was no place in the UI for you to adjust these kinds of settings. Basically whenever an object needs a specialized interface, it will appear in the Property Editor when you select the object.

Icons Found

2008-11-01T17:59:00.000-07:00

Special thanks to Jean-François Fortin Tam, (Nekohayo on IRC) who submitted some icons for me to use on the new toolbar buttons. Any feedback would be appreciated.

Icons Needed

2008-10-29T22:52:00.000-07:00

My refactoring of the UI is nearly complete, so I've started adding new features to PiTiVi. Today I got the Unlink/Relink features working. Normally audio and video clips are linked together. Now you can unlink them and manipulate them separately. Relink allows you to reconnect a source to it's mate if you Unlinked them by accident. I have a couple more features planned for the near future, including: "collapse", which eliminates gaps in the selection; and select-before and -after, which select all sources right or left of the currently selected source(s).

This means we'll need some artwork to help represent these features. I'm using stock icons for now, and as you can see, it's not the best fit. The razor tool is easily mistaken for the "cut to clipboard" command, and the it's not at all clear what on earth "connect" and "disconnect" do. Anyone interested?

Master List of Icons

Other things on the horizon include thumbnail previews for sources, and audio waveforms for audio files. After I get through that, I'll start working on a keyframe-curve for audio-volume.

Sorry, it's been a while

2008-10-24T14:51:00.000-07:00

Good news! I'm now a Collabora employee, hired part-time to hack on PiTiVi. We have set a deadline for this coming April to finally have a usable release. It's rather ambitious, considering how long this project has been languishing, but I think we're finally in a position to really make some progress. Now that we have basic cutting and trimming, we're moving on to the more challenging features. The following is a list of the user-visible features we want to have working by this April.

moving/trimming multiple items
support for still images (along with zooming, croping, rotating and scaling)
linking and unlinking clips
multi-layer editing...
..which goes hand-in hand with audio mixing and video compositing...
...as well as basic video effects.
undo/redo support

This is the big push, and unfortunately, it requires more back-end work. At least this time, I'll be able to help out with a good portion of it, since a few of the changes are based on my own ideas. The refactoring will enable the features listed above, as well as support some long-term goals:

Multi-track editing (i.e. multiple named output channels)
Subsequences and subcompositions
Combining media media from multiple sources into a unified objects (syncing separate audio and video, for example, or combining video from multiple cameras into a single object)
Converting between media types, like converting an audio stream to a video stream with a visualizer plugin.
Pre-rendering and caching of data.
Re-conforming a project between different resolutions, codecs, and container formats.

I have already started work on an improved UI framework, which is almost complete. When the existing interface is ported over to it, we'll be able to add new features more easily than at present.

We're also adding a new UI component: the property editor, which will allow the user to tweak per-object properties. This interface will also be extensible: modules and plugins can provide appropriate controls for a wide range of editing tasks, from chroma key to animation.

We're getting there, slowly but surely.

Anonymous Comments are moderated

2008-09-13T22:07:00.000-07:00

They will not show up right away. Only one submission is necessary.

An argument for removing the simple timeline

2008-09-11T11:41:00.000-07:00

The problem with the simple and advanced time-line is that the two UIs present different and incompatible abstractions to the user. I'll give you an example. The simple timeline allows for setting clip volume through a slider. Presumably, we want to set a constant volume. The advanced timeline will ultimately allow you to set the clip volume to an arbitrary interpolation curve. These are two different, incompatible representations for volume.

Imagine that a user has spent countless hours tweaking the volume key-frames for a clip. What, then, happens if they switch to the simple time-line. What value does the volume slider display? What happens when the user moves the clip's volume slider? Does it flatten the curve? In that case, should we bring up an obtrusive dialog to warn them against what they are doing?

The easiest solution I can think of is to drop the volume slider from the simple time-line (making the simple time-line much less useful). A better, but harder solution is to keep track of whether key-framing has been enabled on a clip, and if so, disable the volume slider in the simple time-line. This creates extra work. A third solution is to prevent the user from returning to the simple time-line once they've activated certain features, like key-framing. This could be frustrating, though, if the user was exploring and the changes were accidental.

An altogether bizarre solution would be to adjust all the slider values based on the current playback position (through property notifications). The sliders would move up and down during seeking and playback as if moved by ghosts (following the interpolation curve). Moving a slider would essentially create and set a key-frame at the current playback position. This would seem to be an ideal solution at first, because it makes the simple timeline equally as powerful as the advanced time-line. Imagine, though, how frustrating this would be for the user: they set the volume in one part of the clip, seek to another, and the volume has changed. It's not obvious why. So, they move the slider back. The user thinks their changes just didn't take, when actually they're editing a completely hidden data-structure. When they go to preview the video, the audio volume goes up and down erratically.

This is exactly the kind of unnecessary problem caused by splitting the UI. You have one application model and two views. Basically it's pretty much impossible to decouple the view from the model entirely -- the two are going to remain highly coupled because the view is a representation of the model. If you try to support two incompatible views with the same model, your model becomes schizophrenic.

According to the late Jeff Raskin (one of the original designers of the Macintosh), the concept of having a basic and an advanced mode is fundamentally flawed:

Psychologist Clifford Nass....reflects on the common assumption that users can be grouped into two classes: beginners and experts, perhaps with a few temporarily in transition. This dichotomy is invalid. As a user of a complex system, you are neither a beginner nor an expert, and you cannot be placed on a single continuum between these two poles. You independently know or do not know each feature or each related set of features that work similarly to one another. You may know how to use many commands and features of a software package; you may even work with the package professionally, and people may seek your advice on using it. Yet you may not know how to use or even know about the existence of certain other commands or even whole categories of commands in that same package. For example, a user of a photo-processing application who produces only online images may never need, or even learn of, that same application's facility for doing color separations, a feature needed primarily by commercial printers.

Interface designers have tried various approaches to accomodate the premise that usres can be separated into beginners and experts. Because this premise is false, the approaches have failed. Adaptive systems that shift automatically from beginner mode to expert mode when they judge your compentance has reached a certain level is are a good example....On the other hand, there is no theory that tells us that the same fixed interface cannot work well over the full span of a person's experience with it, from novice to old timer. It seems best not to have to shift paradgims during your use of the product, and no elaborate analysis is needed to reveal the advantage of having to learn only one interface to a task.

It is easy to fall into the trap of designing different interfaces for different classes of users because by doing so you can make sweeping assumptions that simplify the design process. Few such assumptions are likely to be true of every user in any reasonably large class of users that you specify. The antidote is to view an interface not from the perspective of a class of users but rather through the eyes of an individual. Every person who uses software over a long period goes through a relatively brief period of learning for each feature or command and a far longer period of routine (and, we hope, automatic) use. We do need to design systems that are easy to learn and understand, but it is more imporant that we make sure that these systems can be efficiently used in the long run.

The learning phase of working with a feature involves your conscious attention. Therefore, simplicity, clarity of function, and visibility are of great importance. The expert phase is predominantly characterized by unconscious use of the feature; such use is enhanced by such qualities as aptness to the task, modelessness, and monotony. These requirements are not in conflict; therefore, a well-designed and humane interface does not have to be split into beginner and expert subsystems. (The Humane Interface, p 68-70)

In other words, if we've done our job right, new users should have no trouble learning the interface (provided they have sufficeint knowledge of video editing). We need to strive to make things clear, simple, and visible. I don't know how much clearer you can get than a proportional timeline in which the length of a clip is proportional to its duration. I don't know how much simpler you can get than click-and-drag to move or resize the clip. We make things more visible by putting all the features of the program out in the open, particularly key-frames, where users can see them. And this system works for advanced users too: by focusing on noun-verb interaction and drag-and-drop, we eliminate modes, and provide monotony, allowing users to get comfortable with the command set.

Future Plans

2008-09-10T14:10:00.000-07:00

This is my personal wish-list of PiTiVi development. This represents my goals for the next 4 months or so of development, assuming a lazy pace:

Deprecate the Simple Timeline, or at least scale back it's scope so that changes to the simple timeline cannot do much damage in the advanced timeline.
Add seeking previews during drag operations
Rudimentary undo support
Editing markers
Still Images
Still Image Sequences
Design Undo/Redo framework

I'd really like to bring some new developers on board. If anyone wants to work on this stuff with me (particularly undo), I'll help you get up to speed on the code base.

The problem with the simple and advanced time-line is that the two UIs present different and incompatible abstractions to the user. For example, imagine that a user has spent countless hours tweaking the volume key-frames for a clip. What, then, happens if they switch to the simple time-line and move the clip's volume slider? Does it flatten the curve? What value does the volume slider display? The easiest solution to this problem is to drop the volume slider from the simple time-line (or at least disable it if key framing is enabled.

Guadec Wrap-Up

2008-09-08T17:31:00.000-07:00

Guadec is officially over now. I have taken the final step of uploading a tar ball to Google's website (this is a formality). The work from SoC has been merged into trunk, but unfortunately problems have materialized in testing. School has started for me which means I am scaling back my commitment of time. I'm hoping to hear from new developers who want to pick up where I left off. A couple of things need to be more fully specified:

Undo/Redo support (which operations will support it), and how to implement it
Back end support for effects and transition

In some respects I didn't get all that far this summer. I started out by trying to simply port both timelines over to goocanvas, and this job consumed the majority of my time. Image support is still lagging, though we are a bit closer to implementing this.

I need to update my design document a bit: I actually want to make the UI a little more general. Instead of a "content-image" portion of each timeline object, I just want to have a "UI" portion. I want to be able to define custom UI for different kinds of media and effects that will appear within each box. The default "UI" will be a keyframe editor, but other UI modules could be constructed. The trick will be in choosing which UI will appear when. Perhaps a factory-based system could be used, mapping factory types to their appropriate UI. This would be good for plugins that want to define new source/effect types and then have UI for them.

The simple timeline broke somewhere between the time I stopped working on it and the end of the summer. I have no idea what's going on, but it's damned annoying. I'm kindof not that interested in maintaining the simple timeline anymore. I'd much rather focus on the advanced timeline, because really, advanced isn't that hard to deal with. The goal of the simple timeline was to be easy to implement, IMHO, and it's actually a little harder to work on than the advanced timeline at this point. But in any case, being able to switch back and forth is going to create complications that I don't really want to have to worry about.

Edward is working on switching PiTiVi and gstreamer over to git (from svn). When that happens I'll start maintaining my own branch where I can hack on some things a little more freely. In the mean time, my future commits will probably be related to bug fixes so that people can actually use the work that I did this summer.

Finally, There's one new feature I wanted to add to make the minimal feature set that the advanced timeline now support a little bit nicer: When you trim a source or drag a source in the timeline, I want the viewer to seek to the relevant edit point. I.e. when you move a source, you should see the point in the timeline just before where that source comes in in the viewer. When you trim a source, you should see the exact in-point that you have set in the viewer. All of this should happen while you are dragging the mouse so that you get instant feedback on your editing decisions.

Choices

2008-08-13T13:18:00.000-07:00

I've been working on the design a bit more. The top part of the diagram is an anatomical description of what objects in the advanced timeline should look like. Ideally I'd like all objects to support keyframeable properties, not just effects. This will require some changes to PiTiVi core -- in particular, all timeline objects will have to support getting/setting keyframes, which means internally they'll all have gst-controller objects. Personally, I think that's fine: one thing people overwhelmingly seem to want is to be able to directly control image opacity and audio volume.

One thing I'm not quite sure how to handle, though, are simple effects. You see, Edward's paper notes suggest that simple-effects (effects which only take one source as input) should appear in a small gutter above the other timeline objects within a given layer. This gutter would also be used for transitions. But this design leaves no room for over-laying keyframes, unless we allow simple-effects to expand to full height. But if we do that it will be difficult to distinguish beween simple effects and complex effects. So, perhaps *all* effects should simply be full-size objects , as in the above design. One thing I don't like about this, though, is that it's actually kinda hard to tell what's going on. Will videobox apply to just the image source directly below it, or the entire composition?

So, I came up with a different design. Move the properties down below the timeline, but keep them in sync with the timeline's scroll position. Now all objects can have as many properties as they want, and they can be kept completely separate from each other (or they could show as overlapping curves, I'm not yet sure which is best. The idea is that you select an object (in this image, the lighter colored simple effect is selected), and the property viewer displays its property graph. If you select multiple objects, one of two things could happen: it could show the intersection of the properties of all selected objects and let you set the same keyframe/values for all selected objects; or the property browser could display the properties of all selected objects individually. Ideally you'd want some combination of both, but I can't think of an elegant way to do that just yet.

Details: In either design, whenever you adjust a property, the viewer should show a preview of the keyframe after the effect has been applied.

2008-08-11T16:25:00.000-07:00

Still thinking about the current problem of multiple selection draging. The only thing I've been able to work out so far is that there needs to be some separation between the event handling code and the widgets presented to the user. In other words, rather than doing Model/View-controller, there really should be separate Model/View/Controller objects. Goocanvas items are View-Controller objects: they represent data in the view, but they also handle user interaction. This is convenient when there's a one-to-one relationship. I wrote most of my code with this assumption in mind, so it's really easy to make any canvas item independently movable. Trying to move several items together poses a problem because each item will have an explicit pointer grab and receive all the pointer events. This is best summarized with the following diagram:

The alternative is to have some other object handle all the pointer events, and set the appropriate properties on the core objects:
I like this idea because it allows the selection to simply be the set of core objects on which the next command will take effect. It allows goocanvas and GTK to do all the event handling, however, it puts all of the logic for user interaction in one place: the signal handler attached to the root canvas item. Why is that bad? because not all PiTiVi core objects are alike, and not all have the quite the same interface. In particular, some objects have media-start/media-duration, while others don't. And if the media/start-media duration are not kept in sync with timeline start/duration, wierd things will happen. It would be great if they did, and this might be motivation to develop a consistent interface for all the PiTiVi core objects (i.e. abstract media-duration into media-offset).

Another way to get around this is to subclass the base UI timeline object and provide a handler which does the actual property setting. I don't really like this because it means that the current selection is a list of UI objects. This means the UI has to filter out the canvas items which are not directly selectable (because many items are actually groups of items).

Still another way of doing this might be to move the notion of selection out of the UI and into the back end.

In any case, the trimming handles on each source are a special case: we don't want to resize all the objects in the selection at the same time (do we?) My intuition tells me that users wouldn't care for that very much, because I wouldn't care for that very much. This isn't the end of the world: one simple solution would be to implement trimming handles as separate objects (as is currently done), and let them handle their own events. This way they won't propagate up to the root item, and therefore won't affect the current selection. And we can do this for any kind of UI element we don't want to become individually selectable, like, for example, objects representing keyframes on an effect item.

Redesign

2008-08-06T16:37:00.000-07:00

Things were plodding along fairly well until I tried to start working on multiple-selection-dragging . I got stuck, seriously stuck. I couldn't find a clean way to handle this. Things got even worse when I started thinking about how I would implement keyframes and editing markers. I decided to go back to the drawing board:
Basically, after taking a few days a few days to deal with personal matters, I started redesigning the implementation of the advanced UI from scratch. I haven't written a single line of code.

The inspiration that I hit upon recently is the idea of a Control Point, which is a generalized mapping between something the user can see and manipulate to an arbitrary property belonging to some object in the model. The idea is that for every property in PiTiVi core that you want to control, you have a control point to control it. In addition, when properties in the advanced UI change, the control points update their values as well. This way, objects in the UI are actually defined either as control points or as sets of control points. All the user interaction will be routed through a central Selection object, and then dispatched to all the selected control points. This supports everything from multi-selection dragging to handling undo/redo.

Where this idea goes wrong is when you have an object defined as a relationship between control points that is not one-to-one. For example, the relationship between the start/duration properties of a TimelineObject and the Start/End points of a ComplexTimelineObject is not one-to-one. If you adjust the start or end point of the object independently, both the start and the duration values change. If you adjust the Start/End points together, only the Start property changes. This is a consequence of the mathematical relationship between the out-point of the object and the source's start/duration. I'm not quite sure where to go from here, but I think the best course of action is to provide a layer of indirection between the control points and properties. If you have a proxy object which has an in-point and an out-point, which in turn computes start/duration, for example, all the UI Selection object needs to care about are the In/Out control points.

2008-07-28T19:03:00.000-07:00

I spent today hanging out with a friend of mine in San Francisco who does 3D modeling, rendering, and video editing for a living. While not a developer, he's a computer savvy user who can deal with complex user interfaces. After watching him use Adobe Premiere and After Effects for half an our, I did a short demo of PiTiVi. I wish I could say the demo went well, but, in fact, it crashed several times. Talking with him was useful because he has used three major commercial editors: Premiere, Vegas, and Final Cut. He's also familiar with Maya, and After Effects. The interview was casual, so I didn't record or take notes on the interview. A brief summary of what I learned is available here.

Revised Todolist

Frame Snapping
Collapse Selection
Editing Markers (see below)
Source Splices
~~Fix Text item clipping~~
Drag multiple selected objects
Select Before Current Source
Select After Current Source
Undo/Redo
Multi-layer editing
Still Images

2008-07-24T15:18:00.000-07:00

~~Make zooming work again~~
~~Magnetic Timeline Playback Cursor~~
Frame Snapping
Collapse Selection
Editing Markers (see below)
Source Splices
Fix Text item clipping
Drag multiple selected objects
Select Before Current Source
Select After Current Source
Undo/Redo
Multi-layer editing
Still Images

2008-07-22T13:00:00.000-07:00

Still images are not completely done in the back end, so they are still at the bottom of the stack. I'll bump them up to top priority once they are done in the back end. I am anticipating minor changes in the UI code to make still images work, but we'll see.

I managed to get the razor tool working today, which is incredibly satisfying. Next on the list is to fix zoom support, which was broken by the layout changes. Once that's done, I think it would be nice if the ruler's playback cursor not only extended all the way down the timeline, but was magnetic as well. This would mean that editing operations could easily be made to "snap" directly to wherever the playback head happens to be, including trimming and razoring sources.

Here's my revised todo list:

~~Split clips in advanced UI~~ (Done)
Make zooming work again (In Progress)
Magnetic Timeline Playback Cursor
Frame Snapping
Collapse Selection
Editing Markers (see below)
Source Splices
make Text item something we can shrink so it doesn't protrude past edge of clip at high zoom (in progress)
Drag multiple selected objects
Select Before Current Source
Select After Current Source
Undo/Redo
Multi-layer editing
Still Images

I want to introduce a feature I'm calling editing markers. My understanding is that some commercial products do this already, so perhaps some of you will be familiar with this idea. Basically, you should be able to drop an unlimited number of "markers" which will serve a number of functions:

visual "bookmark" of some important point in the timeline
magnetic edit point for spliting, trimming, and moving sources.
general input to editing commands: For example, if you select n edit points and n sources, and then choose the razor tool, a cut will be made at the edit point for each source.

Initial Design

Place edit points by clicking the marker button, or pressing m. The edit point will appear as a small triangle beneath the ruler at the current playhead position.
You can fine tune edit points simply by draging them
You can select edit points as you select any other timeline object
When you activate the razor tool, drag a source, or trim a source, the effective cursor position will snap to the edit marker if the cursor position is within the edit marker deadband.
If any edit points are selected when the razor tool is active, the razor tool will treat the edit points as input coordinates. Other commands may operate like this as well.

2008-07-20T20:45:00.000-07:00

See previous posting for current progress. Just wanted to post a screenshot of the new layout:

The orange bar is visible because the razor tool has been activated. Razor tool doesn't actually split the sources yet, but I'll probably get that working tomorrow morning.

2008-07-20T11:31:00.000-07:00

Here's my revised todo list:

~~Fix graphic bug created when advanced timeline loads from file which places clips in the wrong order.~~ (Done)
~~Layout changes, implement a toolbar~~(done)
~~Delete clips in advanced UI~~ (done)
Make zooming work again (=P)
Split clips in advanced UI (In Progress)
make Text item something we can shrink so it doesn't protrude past edge of clip at high zoom (in progress)
~~make edge snapping work~~(done)
Drag multiple selected objects
Undo/Redo
Multi-layer editing
Still Images

The Vision

2008-07-17T15:19:00.003-07:00

After a vigorous discussion on IRC this morning, my goals for the advanced timeline have crystalized in my mind. I'm shifting from my original focus on the simple timeline, and I am going to concentrate on basic editing in the advanced timeline. I thought I'd share my plans, since currently there are no mock-ups for the Advanced Timeline in any form. I wanted to take the time to include some sketches, but I'm getting tired and I want to get my thoughts out.

Design Principles

Use direct manipulation for most common operations.
Use noun-verb pattern for most other operations: select first, then click a button in the toolbar to perform a command.
Minimize the number of modes. Modes are evil. Use context to decide which action is appropriate.
Constrain actions to sensible values by default. Provide modifiers to override these defaults easily.
Provide feedback. Change cursor styles, provide real time updates.

Most common Tasks

navigating the timeline
moving sources
trimming sources
adjusting edit points

Other Tasks

deleting sources
moving large numbers of sources
grouping multiple sources into single source
decoupling audio from it's associated video

Layout

Ruler at the top
Sources in the middle
Tools at the bottom
Tools include: split, select-right, select-left, delete, unlink, group, ungroup,collapse selection,zoom adjustments,

Keyboard

shift, ctrl, alt, and ctrl+alt act as quasi-mode modifiers

Design Features

Only conventional "tool" is the split tool. By default, you activate it first, then click where you want to cut. Later, I want to implement an alternate way of making cuts with markers, but people are already familiar with the current method, and in this case it makes sense to have a mode because it mimics the act of physically cutting a piece of film. After one cut, the tool resets.
Delete, Unlink, Group, Ungroup, Collapse, all operate on current selection. These commands are only sensitive when valid on the current selection.

Selections

By default, click or click-drag to select.
Use shift to take the intersection of the current selection and the new selection.
Click-and-drag on the current selection will drag every object in the selection.
Two tool bar buttons will allow the user to extend the selection down the timeline to the right, or to the left, of the currently selected source.

Operations

Group/Ungroup combine multiple sources into a single source. If the selection includes audio and video, two groups are made (one for audio/video).
Unlink allows for dissociating linked audio and video tracks.
Collapse arranges all sources in the current selection so that there are no gaps between them. (perhaps a leftward /rightward variant should be provided)

Edge / Frame Snapping

When moving sources, sources snap to the nearest frame at the clip's native frame rate (depending on zoom), or the project's current framerate if the clip does not have frames. This can be overridden by holding a modifier key. Ctrl will disable frame snapping entirely, while alt will switch to the project's native frame rate.
User can trim a source at any time by clicking-and-draging the trimming handles at the ends of the source widget. A source cannot be stretched beyond its actual duration (time stretching will be implemented later, as one of those evil modes). By default, trimming snaps to the clip's native frame boundaries, but this can be be over-ridden with shift/alt/control as described above with moving sources.
The edit points of sources are "magnetic" within a certain distance. This means it should be really easy to make one source cut directly to another, despite the default frame-snapping. Care must be taken that the snap-to-frame feature doesn't interfere with this feature: it should take priority.

Fine-Tuning of Edit Points

When the edit point of one source coincides with another, the clips are automatically "spliced", meaning their in/out points become linked together.
When two clips are spliced, the area between the two sources works as an edit-point adjuster. Essentially it adjusts the out point of the left source and the in point of the right source at the same time. A bi-directional arrow cursor <-|-> should be used to indicate the presence of the splice.
By default, edits can snap to the frame boundaries of either clip. The user can override this with alt/control as described above.
The splice can be broken by moving the clips or trimming them as usual: only the area in the center provides this feature.
The UI should constrain the edit point in such a way that neither source can be stretched beyond it's native duration.
Note this does not address the (possibly more common) scenario in which the user wants to adjust the in / out point of only one of the sources, but still keep the hard cut between them. In this case, the user first performs the trimming operation, then uses the collapse command to eliminate the bank space created.

Multi-Layer Editing

The user can add additional layers to the timeline. For the moment, these will merely be an organizational convenience, but later they will play a role in inputing effect operations.
By default, audio will be mixed, and the topmost video source will play.
Future releases will introduce compositing support.

Seeking

Scrubbing directly on the timeline ruler will seek to that place in the timeline at the nearest frame in the current project settings.
The user can use forward and back arrows to seek forward/back single frames at the current project framerate.
Holding down keys causes repeated seeking. Holding shift increases the seek interval to 5 frames. Holding down ctrl increases the interval to 1/2 second. holding down alt increases the interval to 5 seconds. Holding down ctrl+alt moves the play head to the next edit point.
If the playhead moves beyond the current scroll position, the timeline scrolls to the keep the playhead position in the center when the user releases a key.

Zooming

Zooms should always center on the current play head position, not the scroll position. the zoom control should provide meaningful zoom levels: 1, 5, 10 frames, 1, 5, 10, seconds, 1, 5, 10 minutes.

My main concern is that I'm relying on modifier keys to control certain parameters: but I think the way I've used them is justifiable. I use them more-or-less consistently, and their function can be explained in a couple of sentences. If the defaults don't turn out to be sane, we can change them, and still use the modifiers to access the other modes. I think only user testing and feedback will really tell us what the defaults should be.

Post-Guadec Update

2008-07-16T23:17:00.000-07:00

Ugh. I hate airlines, Delta in particular. Coming home was a real nightmare. I came back to find that my homebrew exploded all over my room, so tomorrow I get to shampoo the carpets. I was only able to salvage about half a pitcher of it, as it had a tendency to leap out of the bottles once the cap was removed. Fortunately, it turned out to be very strong.

I committed changes which implement resizing of clips in the advanced timeline this morning. So there's a pitivi first. I'm tinkering with support for "edge" snapping, which is crucial for usability. Edward is still working on refactoring pitivi core so that we can handle still images. I also added some cosmetic changes, because I was tired of the way things were looking. Unfortunately, the only thing giving you any clue you're resizing something, is the cursor, and that didn't come through in the screenshot.

Here's my revised todo list:

Fix graphic bug created when advanced timeline loads from file which places clips in the wrong order.
Layout changes, implement a toolbar
Delete clips in advanced UI
Split clips in advanced UI
make Text item something we can shrink so it doesn't protrude past edge of clip at high zoom
make edge snapping work
Drag multiple selected objects
Undo/Redo
Multi-layer editing
Still Images

I've added Undo/Redo support: now that pitivi can do some dangerous things, undo/redo are crucial. I doubt i'll get to that by the end of the summer, though. Still images are at the bottom of the stack, because at the moment they are impossible to implement. When Edward fixes PiTiVi core, I'll move them up.

In the mean time, i'm thinking about how undo/redo support should work. It's kindof a tricky problem. One hackish way to implement it would be to serialize the timeline to the intermediate format after every major undoable event (via a signal emitted from PiTiVi core or in the UI). If the user wants to undo, then we can simply reload the undo tree, as if we were loading a file. That might be a way to implement single-action undo/redo for the next release. Multiple undo/redo will have to be a little smarter: a stack of operations needs to be mantained somehow, and each operation needs to have an inverse. Undoing in this case means poping the top of the stack, and applying the inverse operation to the timeline. The trick of course, is providing the infrastructure for both maintaining this history, and reversing the changes. Part of the problem is deciding which events are actually undoable. For example, during the course of a drag in the PiTiVi timeline, a source's position will be updated potentially thousands of times. We only care about its position before and after the drag operation, and only the UI can provide this perspective.

I wish I had some kind of handbook on how to implement these big, complicated, yet crucial features.

Guadec Update

2008-07-11T08:21:00.000-07:00

Edward and I worked out the following todolist for me:

(done) Fix file load/save support
(done) Multiselection in advanced UI
Layout changes
Resize clips in advanced UI
Deleting in advanced UI
Split clips in advanced UI
Multi-selection Dragging
Time-stretching of clips

As you can see, the first two items are done. The next step is to make some layout changes to the advanced UI to allow for tools, such as a cutting razor, shifting of sources, what have you. The first tool will be a delete button, which will delete everything in the current selection. Then, I'd like to have a trimming razor which will split the cursor at the mouse position (some kind of floating bar will show where the source will actually be trimmed, an alternative would be to trim at the current playback head position).

Meanwhile, Edward is working on adding still picture support in PiTiVi core.

The first step is a bit of refactoring to move useful code into more abstract classes.
A custom freeze-frame element will be created for actually handling still pictures within a timeline.
This will be used to implement a TimelineImageSource object in PiTiVi core.
Finally, changes to the discoverer will be necessary to avoid long delays when importing large images.

So, by the time the PiTiVi backend can handle images, the ComplexTimeline should be a lot more functional. And, finally, you can actually save your work.

Even at GUADEC, Slow Going

2008-07-08T14:19:00.000-07:00

GUADEC is a lot of fun, but the network at the venue is a bit dodgy. Worse still, a number of useful ports are blocked. This has complicated just about everything. Also, I tried to update my gstreamer install and the cvs-update.sh script barfed all over everything, borking my cvs install of gstreamer. I might have it fixed by tomorrow morning =(

The good news is that Edward and I fixed a couple of rendering bugs in the advanced timeline. The key to the problem was the GTK+ debugging flag --gtk-debug=updates, which shows you which areas are being redrawn (though I can't yet use this on my machine, still need to recompile the gtk+ library to enable debugging).

Also, I finally got to look at Edward's design notes for the Complex UI. I finally have an idea of what I'm building. There's still a lot to discuss, but Edward and I haven't quite found time for a hacking session.

That's about it for now.

Bugs

2008-07-07T02:42:00.000-07:00

Apparently there are some bugs in the latest goocanvas code which cause the timeline to update portions of the display which are much too large. I'll be looking into this at some point this week. Also, a fellow contributor has submitted a patch to add a vertical playhead bar to the timeline.

Update

2008-07-04T00:27:00.000-07:00

I lost a couple days due to a stomach bug of some kind (too much street food in Istanbul?) I've been using some of this down time to do research. GUADEC is next week, and I expect that Edward and I will get some serious hacking done during that time.

I've been meaning to interview some of my video professional friends to see how they use existing video editors, and find out what they like and don't like about them. In preparation for that, I have been reading on-line tutorials for existing video editors to get an idea of what expert usage looks like. I like the user submitted tutorials better than official documentation, because it helps me get into the frame of mind of a user, rather than a designer. Tutorial explanations hilight the how users conceive of the problems they are trying to solve much differently than designers. Often they use tools originally intended for a different purpose to achieve their ends. The downside, of course, is that tutorials are mainly written by expert, tech-saavy users. Right now, the main thrust of PiTiVi's development has been targeted at novice users, or at least users unfamiliar with the domain of video editing.

Part of me wonders whether or not this is a mistake: since PiTiVi is a linux video editor, relatively few novice users have access to the application. And we already have Kino, which is more mature than PiTiVi. If I were a novice user trying to get video editing done with Linux, Kino would be my only real choice. On the other end, we have Cinelerra, which seems to offer more high end features (though I've never gotten it to work). Maybe PiTiVi should aim for the middle-of-the road, intermediate crowd. Instead of dumbing down for 5 year olds, or making some ridiculous attempt to out-maneuver FCP (not going to happen), we shoot for an even balance of features: more precision and control than Kino, but don't plaster the screen with controls and widgets. At the same time, we should provide an environment that will be familiar to anyone who has used Premiere, or FCP, or even iMovie, so that their skills transfer. This will fit well with features that gstreamer already provides: an open format timeline, and real time processing of transitions and filters.

PiTiVi could fill a real niche in the amature filmmaking crowd: we can support the odd-ball movie formats that modern digital "still" cameras, like Canon PowerShots, spit out. If PiTiVi makes using these cameras as simple to use as other editors make DV, it will be an attractive tool for people making movies on a shoestring budget.

The existing simple timeline code might be re-imagined as something similar to FCP's media browser: useful for setting rough edit points on clips not yet imported to the timeline, and we needn't worry about providing support for effects or transitions.

Relevant Tasks:

clip logging and renaming in the library
import movies from "still" cameras
"off-line" (low-res) editing and recapturing (high-res)
synchronizing audio and video from different sources
animation and timelapse from sequences of stills

For next week, I want to take advantage of the time I have with Edward. He has much more in-depth knowledge of gstreamer and pitivi core, so I'll be taking a break from the UI work to solve some of the back-end problems preventing things like still images from working.

ToDo List:

fix the freeze plugin, or write a replacement, so it works with gnonlin
implement transition and effect objects in PiTiVi core
implement still image sources in PiTiVi core
discuss the complex UI design (please bring your notes, Edward)
(optional) make other mixing modes in videomixer available through property
(optional) tinker with gst-editor a bit, see if it can be made more stable

Starting to Look Good

2008-06-30T06:31:00.001-07:00

The goocanvas version of the advance timeline is taking shape. The above screen shot shows a timelinew with a single source zoomed in. I haven't tried messing with multiple sources, but it shoul work alright. This revision is a bit of a regression, as the timeline now lacks the ability to trim sources, but you can move them. The timeline is also miss-aligned. My gut tells me that it's in the wrong spot to begin with, and the best thing to do would be to drop it down below the toolbar. This would leave more room for other tools, like a trimming tool.

Now is where I need your help: color schemes, layout suggestions, usability concerns. The current source widgets are just rectangles, do people want thumbnails or just text to cover those? How should effects be represente visually? I don't really want to go too much farther without a good idea of what people want to see.

Got something done! Yea!

2008-06-22T09:00:00.000-07:00

Testing screenshot showing a ComplexTrack with several sources, two of which are overlapping. Sorry about the colors: I'm a programmer, not an artist.

I'm currently in Prague, which is a gorgeous town. Primarily due to lack of planning, I've got a couple of dead days which I used to work on PiTiVi. I had left the States with out a complete gstreamer install, without svn installed, without goocanvas, and a whole host of other silly things that are easy to take care of when you have reliable internet access, but impossible without it. Today, after two weeks, I have finally made another commit.

Basically, I figured out the easy way of making some of making some of my utility functions a bit more friendly to MVC. I have a function calle make_dragable which works on any goocanvas.Item object. It connects signals and sets appropriate data items to make the canvas item dragable. Previously, this function and its associated callbacks handled all the interaction from button_press to button_release, including actually moving the object in question. This posed a problem for the MVC design pattern, as the object should not actually be moved until the appropriate callback is received. I wanted to save as much of this code as possible, as it handles a lot of nice details, like keeping track of the mouse-down offset (so that the object being moved doesn't suddenly jump to the cursor position). I had tried various ways around the problem, such as having a flag to not set object position, using part of the code and connecting signal callbacks manually, even starting over from scratch. Everything I tried had problems from breaking existing code to just being downright buggy. There's no substitute for already working code.

I finally hit on the solution in the shower yesterday morning, and it involved changing about 3 lines of code to add an extra callback. Worked like a charm. I can finally get moving again. This will also make the simple timeline code easier to manage, so though I haven't written many lines of code in the last 2 weeks, I have actually made a good deal of progress in terms of thinking about the problem.

Update

2008-06-18T01:19:00.000-07:00

Well, i've been a bit preoccupied with travel preparations and the traveling itself. I'm also finding development on the eeepc a challenge, partially internet access was hard to comeby. I'm slowly chipping away at the problem, working bout 45 minutes a day. I haven't got anything commit-worthy yet, i'm still trying to sort out some issues. Also, a lot of the gstreamer encoding plugins failed to build, and the sample video i brought is a little too intense for the eeepc. I can't re-encode it, though, without the encoding plugins. Anyways, I'll post another upddate later this evening. I have a long train ride ahead of me tomrrow, so I ought to get a lot of work done.

Started Porting Advanced Timeline

2008-06-08T09:38:00.000-07:00

While it is true that the advanced timeline presents a simpler visualization than the simple timeline, the simple timeline is still simpler: it only represents the video composition of a timeline, and relies on the fact that video and audio sources are linked to each other to make it appear as though sources are complete movie files. The complex timeline presents a simpler visualization, but it might manage dozens of instances of each. So, my initial optimism has begun to fade somewhat. I'm going to just focus on re-implementing existing functionality, while also adding support for resizing clips which currently can't done. Transitions and effects will be easier to implement in the simple timeline first.

Did some light refactoring yesterday, eliminated a redundant class, and restructured the simple timeline code somewhat so that it will be easier to add elements to it that aren't scrolled along with the timeline. The structure of the simple timeline now follows the structure of the complex timeline more closely. I've been studying the advanced timeline code to better understand what has to change, but I'm still confused about a few things.

I've started working on a ComplexTrack() class, based on SmartGroup which emulates the existing timeline widget pretty closely. I could easily spawn separate instances of the class to connect to the video and audio compositions. The real issue is that pitivi emits separate signals for effects and transitions. I'm wondering if another layer of indirection might be needed, but of course, a lot of this depends on just what the complex timeline is supposed to look like. Will effects overlap sources? will transitions appear on on a separate track? My primary goal is just to replace the existing layout code with goocanvas, then go one step further and add trimming support (as opposed to mere cutting support, which splits clips in half). What will be the semantics of multiple tracks in PiTiVi? Priority? Nothing at all?

One thing I've realized over the last few days is that the scale_x property is not going to work for handling zooming. It's a great way to handle resizing the simple timeline, but if I use it to provide zooming in the complex timeline, there will be side effects. All the images, fonts, outlines -- everything in the canvas -- will be stretched or compressed by scale_x, which will result in unreadable text and distorted images at higher levels of zoom. Instead, I'll retain the existing zoomable widget interface (possibly in a slightly modified form) and use it to adjust scale "manually". One change I would like to make, however, is in the zooming widget itself. I'd like to provide meaninful levels of zoom, instead of the "bigger-or-smaller" caveman controls we have now. The sequence would go something like ... 1-frame, 5-frames, 10-frames, 50 frames, which are meaningful for movie projects, as well as .1s .25s .5s, 1s, 10s, 1m, 10m...The frame adjustments would be based on the current project framerate, and would essentially make each quantity of frames a standard width. For video projects, it would also be nice to constrain the playback head (red line) by 1/FPS milisecond for precise edits.

There are two custom canvas items that would be good to have in the complex timeline, but for now I'll just use the existing widgets: the first is the top level ruler, and the second the thumbnail viewer. For the complex timeline, though, I want more than that: I want a "filmstrip" widget, one which presents a sequence of thumbnails, the number of which is proportional to the visible width of the the widget.

Oh one more thing...I've been sloppy about the distinction between widgets and canvas items. There is a big difference, so I'm going to be more careful about this in the future. In general, though, they are equivilent because you can embed gtk.Widget objects in a canvas, and goocanvas.Canvas() objects are gtk.Widget objects. You can wrap one in the other, essentially.

2008-06-05T21:43:00.000-07:00

Simple timeline is ported over to goocanvas, at least to the point where it's usable. There are some graphic glitches that I'll fix later, and I want to improve the code which handles external drag-and-drop (currently sources are always appended, I'd like to have some interaction which allows you to drop sources anywhere). I'm switching my attention to the advanced timeline.

I'm probably going to gut all the existing timeline code completely, because most of what's there is redundant with the facilities goocanvas provides. For example, zooming is easily handled by goocanvas's scale_x property. In addition, scrolling is taken care of automatically if you use a gtk.ScrolledWindow, so the layers of indirection are really, really unecessary. I can handle the informational messages directly in goocanvas...basically the structure of the timeline will look like this (more-or-less, parent classes in parens, trailing * indicates more-than-one).

* Generic timeline widget (gtk.ScrolledWindow)
**SimpleTimeline (goocanvas.Canvas)
*** SimpleSourceWidget*
**ComplexTimeline (goocanvas.Canvas)
*** PiTiViTrack*(smartgroup)
**** PiTiViSourceWidget* (goocanvas.Rect)

Conversion of SimpleTimeline to Goocanvas almost complete

2008-06-02T19:39:00.000-07:00

I solved my conceptual dillema by introducing Yet Another Layer of Abstraction: I subclassed HList() to create TimelineList(), which harbors all the application-specific code. This required reworking the HList() class a bit, but it still works as it did before. I even manged to get rid of some code I didn't like in the process. So, now the mouse events trigger changes to the underlying composition, and the UI is only updated in the callback. All that remains now is to handle the external drag-and-drop events (of FileFactories onto the timeline) a little better, so that the user can see where their source will end up before making the drop.

I asked about my pointer-event propagation problem on the goocanvas developer mailing list. It turns out that the only way to get signals from goocanvas.Widget items that contain a widget with an xwindow is to connect to the widget itself, rather than the goocanvas.Widget item. This will require conversion of the coordinates. That might not be so hard, so it's worth pursuing. The current "ghostly" SimpleSourceWidgets are rather distracting, but it's low priority for now

After that, I'm going to start porting the advanced timeline over to goocanvas. This should be relatively straightforward. Each source in the timeline will have a goocanvas.Rect() object, or perhaps a derivative, which will represent it in the timeline. There will be a direct mapping from time to position along the x axis. Zooming will be handled by adjusting the scale_x property of the canvas, and scrolling will be handled automatically by a gtk.ScrolledWindow (that will also contain the SimpleTimeline, and handle scrolling for that as well). Once that's done, PiTiVi will basically work as it did before, except that now it will be easier to add new features using goocanvas. By this time I hope to have development versions of gstreamer built, because I'll be on my way to europe.

Setbacks

2008-06-01T22:43:00.000-07:00

Today worked on the SimpleTimeline using Goocanvas. It wasn't easy: a number of unexpected obstacles presented themselves. The first was the challenge of merging the existing SimpleTimeline code into the new code, and getting drag-and-drop to work as it did before. Once I had done that, I was able to add and delete the existing source widgets from the timeline.

Erhm, well kindof. Actually this is a transient glitch, it goes away as soon as you resize the timeline.

See, now that's better. Except once I got to this point I found that the re-orderable list that I'd spent so much time on didn't work. Event signals never make it to the canvas. They seem to be trapped by the widget. I modified my test code to use widgets instead of native canvas items. A little experimentation showed that only widgets which don't have their own xwindow (as do gtk.EventBox and gtk.DrawingArea) will emit events within goocanvas. So, if I just change the parent of the SimpleSourceWidget to something like HBox, it should work...right?
Well, sortof...now they look like "GhostSourceWidgets"...wooooooooooo! Hmm, I know. I'll make a ghetto hack! If I just plop a goocanvas.Rect behind the widget, and group the two together, then I'll have an opaque background that will prevent widgets from blending together...
Ermmm...no.

Progress

2008-05-31T18:08:00.000-07:00

I thought I'd post some snapshots of the new timeline code. I wanted to test the dragging code independently of PiTiVi, so it is being run from an external script.

Timeline displaying some test items at their default size.

The Timeline can be enlarged, in which case the items inside are automatically enlarged. This takes advantage of goocanvas's scaling.

The timeline is now fully reorderable, though the interaction is far from perfect. It's not as smooth as it was in my early prototype, but my early prototype didn't handle items of varying widths. I'm anxious to move on, but I really want to make sure that this code works properly on its own before I throw in the added complexity of MVC.

One issue which I haven't yet decided how to handle is transitions. Should I allow multiple transitions to be moved next to each other on the timeline? What would that mean? I don't think there's a use case for multiple transitions in between two clips. If I'm not to allow multiple transitions, then I have to think of a way to prevent them from stacking against each other, which could get hairy.

Currently I have a SimpleTimeline which derives from Canvas(). This in turn uses a more general HList(), which I had hoped to make as general as possible. However, I think my goal of having a generic, re-orderable list class is not going to work. There is too much application-specific behavior in the timeline. Tomorrow, I'm going copy the list code directly into the SimpleTimeline class, which already implements the re-orderable portion of the list anyways.

Implementing Reorderable List

2008-05-30T01:07:00.000-07:00

pitivi.timeline.composition.Composition provides the following relevant methods:

getSimpleSourcePosition
addSource
appendSource
moveSource
removeSource
insertSourceAfter

There are also these relevant signals:

condensed-list-changed
source-added
source-removed

The goal is to write code which uses

calls insertSourceAfter() when an user drops item to timeline
calls removeSource() when user deletes item from timeline
calls moveSource() when user moves a source to a new position
responds to the "source_added" signal by creating a new source widget and adding it to the timeline
responds to the "source_removed" signal by removing its widget from the timeline
responds to the "condensed_list_changed" signal by updating items to their new position

User Interaction: Two Approaches

When the user clicks on an item in the timeline, the source will be moved to the top layer of the canvas (so that it shows above other objects as it is moved. The user can then drag it freely in the x direction within the bounds of the canvas, but it will be completely constrained in the y direction.

If the object is moved beyond a certain threshold to the right or left, the position of the nearest object to the one being moved will be swapped (i.e. the order of the objects will change, but the object being dragged will still be under the control of the mouse). The operation finishes when the user releases the mouse.

Two ways to do this:

When the threshold value is reached, simply call moveSource() to exchange the positions of both sources. The advantage of this approach is that the UI updates automatically when the composition emits the "condensed-list-changed" signal. One possibly negative side-effect is that the object the user is dragging will suddenly "jump" in to position. Another potential side-effect is that the "condensed-list-changed" signal might be delayed for some reason, leaving the timeline in a sorry state even after the user releases the mouse.
When the threshold value is reached, only swap the sources visually. Save the proposed changes until after the user releases the mouse. Then call moveSource() to move the source. This avoids the "jumping" side effect, but introduces a different problem. The visual timeline and back end will be temporarily inconsistent. If the "consensed-list-changed" signal is delayed, then the user's changes are incorrectly displayed. We can get around this by automatically restoring the list to its original state after the user releases the mouse. This could also be confusing: the user sees their clips move into a new position, then move back to their original position after releasing the mouse, then move back to the new position after the signal arrives.

I guess I'll just have to pick one approach to try and see if my fears are really grounded. Any feedback would be appreciated.

My goal for my first commit is to come up with a drop-in replacement for the existing timeline. But, in the interest of being forward thinking, there's a design constraint here I almost forgot to consider: Transition widgets. Handling child widgets of different sizes and aspect ratios will not be so hard, but transitions are only supposed to go between two sources. How can I easily enforce this constraint? Maybe by fiddling with the threshold values, but remember that transitions are optional. I could use a mandatory space between sources, but to be honest I really hate that idea. I'll sleep on it for now. One thing at a time.

Distro Shopping

2008-05-29T18:17:00.001-07:00

I spent the day distro shopping, but since I have not really found what I wanted, I have decided to stop worrying about it for the moment. Ubuntu's support of GStreamer isn't that bad. I have other beefs. On the other hand, the my criticisms of Ubuntu also apply to just about every other distribution out there. It's just that Ubuntu initially set the bar very high, and I have been really disappointed with the last two or three releases. I wait and wait, but the functionality doesn't improve, and even regresses. It's very frustrating. I figure that if I'm going to have to tinker with the OS, that I might as well use something much less restrictive. Something that really gives me what I want.

That having been said, enough of this. I can do a fair amount of PiTiVi development without up-to-date gstreamer installed. I have a job to do, and it's time to start doing it. So, for the next three days, I'm going to start working on things which do not depend on having up-to-date gstreamer libraries.

Switch Simple Timeline To Goocanvas

I've already got a prototype of this working, but there are a couple things that need to happen first.

I have a prototype container class that derives from goocanvas.Group which handles repositioning, adding, and deleting groups of items that I use in gst-editor.py. I want to re-factor this class so that it derives directly from goocanvas.Canvas.
I have a prototype timeline implemented as a python script. It simulates a re-orderable list of canvas widgets. This code needs to be rewritten slightly so that it is more friendly to MVC. In particular, it needs to wait for signals from the model before anything in the UI is reordered.
Merge the code from (2) above into (1)
Test code independently with standard canvas objects of varying sizes
Merge the external drag-and-drop support in from the existing PiTiVi timeline widget
Test the code independently of pitivi using existing PiTiVi source widgets
Merge the new code into PiTiVi

Future Goals
Once I get gstreamer-cvs installed, I can start working on some other things.

Transitions
Titling Support
Still Image Support

Gah! Paperwork

2008-05-29T14:30:00.000-07:00

No sooner do I have all my European flights and hotels booked than I realize I hadn't registered for guadec. And now registration is closed...Meanwhile I'm behind on sending in the tax forms to Google. I've moved out of my old cooperative and into this new one that doesn't have a house laser printer. I dug out my old inkjet and spent an hour struggling with CUPS to get it to print only to find that the ink has gone dry. Meanwhile I'm trying to send in the DMV paperwork for my two motorcycles that I'm selling. Combine that with the joys of moving, unpacking, throw in a visit to an ailing mother in a different town, add a spontaneous, free ticket to a rock concert, and you get a Summer of Code student who hasn't been able to write any code for two days =(

Meanwhile, Ubuntu has put the nail in the coffin in the form of an "update" they released today which uninstalled devhelp and prevents my reinstalling it. Seriously, Ubuntu is going downhill. I am done with their silliness. The real question is where to go next. Maybe gentoo, maybe arch, maybe something else.

Wow! Comments!

2008-05-29T14:25:00.000-07:00

Thanks, those of you who replied to the last posting. I agree, installing gentoo *on* the eeepc would be problematic. It just seems that, since eepc's are mostly the same hardware, that there ought to be an disk image of a functioning install that I could just dd onto my hard drive. Then all I have to worry about is setting up my development environment. Cross compilation isn't really a good idea because it seems like it would leave me without the ability to install software when away from my primary machine.

With regards to the SSD hardware, I have heard that it will take quite a few write cycles before one starts having problems. Unless you know about someone who killed their EeePC by trying to install gentoo, I'm not really worried about anything except running out of disk space. And I

I promise to stop hacking and get to work

2008-05-27T23:01:00.000-07:00

For the last couple of days i've been hacking pretty heavily on my gst-editor, another gstreamer project closely related to pitivi. I've made some serious progress in terems of UI, but there are some wierd bugs that have basically led to an impasse. In any case, it's time for me to start work on PiTiVi in earnest. Working on gst-editor has taught me what I need to know about gstreamer and goocanvas.

There is one main barrier to working on PiTiVi, and that is finding a reliable and efficient way of setting up a gstreamer development environment. Now that Edward has written his QA system for gstreamer, a large number of bugs are being fixed. This has led to rapid changes in gstreamer, which is on a monthly release cycle anyways. It's much too fast to rely on a distribution like Ubuntu to keep pace. I have been tinkering witht he idea of installing Gentoo or Arch linux, especially on my eeePc, which I'll be taking with me to europe. Holding me back is the knowledge that configuration will take a long time. I have grown used to the "it just works" nature of Ubuntu. For the most part, I want other people to make choices for me. There are only specific areas in which I want to override those choices: gstreamer, power management, network management. I don't care who makes the choices, as long as it works. I am suspicious that ubuntu doesn't include a complete gstreamer distribution, but I don't know how to check. Do I contact the package maintainers? Even if I build my own packges, I'm still limited by the dependencies that ubuntu includes. If there's some plugin which uses a library that ubuntu doesn't support, then it doesn't matter if I go to the trouble to build it myself.

Chicken an egg. I really wish I had a local gentoo guru who could get me past the initial setup quickly, so that I could concentrate on my work and not the system configuration. When I was using debian, nothing ever worked right. I can only imagine that gentoo would be similar in terms of leaving me a lot of work to do in order toget a user experience similar to what Ubuntu offers. On the other hand, ubuntu is p*ssing me off as of late: their power management system as of 8.04 is terrible, it's bloated, thunar *never* seems to be able to unmount disks, and 8.04's kernels don't support eeepc hardware out-of-the box. I had to go out of my way to set the machine up in the first place. If have to tinker with ubuntu, maybe I might as well just install a more flexible distribution. Sigh....

eeePc

2008-05-23T21:36:00.000-07:00

I got an eeePc for traveling this summer. The following image documents a watershed moment: PiTiVi running on an eeePc. As you can see, there are some screen size issues to work out. But it does work, and I was able to make some editing actions. The eeePc seems to have some trouble playing back the high-res video I took from my friend's cannon powershot. After about 15 seconds, playback becomes jerky. This is in both PiTiVi and Totem. I'll try the same video on my thinkpad to make sure the file isn't corrupt in some way. In any case, I'll have to re-encode the video at a lower resolution / higher compression to save space. Hopefull the playback issues will be taken care of at the same time.

Corrections

2008-05-21T19:37:00.000-07:00

Turns out a couple of the things I said about goocanvas don't make sense now that I've reread them. Goocanvas provides all kinds of support for dynamically resizing and repositioning individual elements: it'd doesn't provide so much support for notifying other parts of the program about those changes. And it only provides rudimentary support for hierarchical nesting of canvas items.

I don't want spend time writing custom canvas items because it's not necessary for PiTiVi: rectangles and text objects will be fine, or I'll embed some existing GTK+ widgets into the canvas. That having been said, I would like to be able to easily compose primitive objects together to make more complex ones: rectangle + text = labled box! I also want to be able to make container classes similar to GTK+s container classes, which dynamically grow and shrink as objects are added or deleted. I'd also like to be able to have the container's size update if any of the child objects change in size. In order to do that, those container classes need to know whenever one of the child objects moves or changes size.

Basically, you have simple items and you have groups. All objects have transform matrices and bounds, but you get no notification if any of these change. You have to use the gproperty interface if you want notifications of changes. This means you can't use the translation matrix portion of the API, since they won't be in sync. Goocanvas groups have no size or position properties either, so I have chosen to add this to a derived class I call a "smartgroup". The issue is that in order to calculate the size of the smartgroup, I need to know the size and relative position of all the children. And now we come to the crux of the matter: since there is no consistent property interface for size and position, I have to make some derived primitives to support this as well. Honestly, this feels like a design oversight within goocanvas. It seems like the stock items should do this for you.

That having been said, goocanvas is great. It's efficient, and the rendered output looks awesome. It's easy to build up the kind of functionality I want, so most of the above should prove to be relatively minor setbacks.

Turns out I HAVE a blog!

2008-05-21T19:18:00.000-07:00

As you can see, I didn't get much use out of this account during the previous year. Well, Here I am again. I've been asked to set up a blog for SoC 2008, and this will do. I'd like to start the summer off with a brief discussion of the challenges that lay ahead for my SoC coding.

Bleeding Edge Gstreamer

The video editor relies on up-to-date gstreamer and gnonlin, sometimes even SVN. Installing these from source is kindof a pain. I tried installing gentoo in a chroot, and using this to build recent gstreamer. This is a great option on my thinkpad, but not so great on my eeePc, which I will be using in europe this summer. So, step one is to get both of my development machines ready to go with bleeding edge gstreamer libraries.

Limitations with Goocanvas

I have been tinkering with goocanvas for a couple of months now. I think it is a very useful library, but there are a couple of issues with it in its current form.

Goocanvas doesn't give a lot of help with interaction. There is a "container" class called Table, but this going to help me implement a re-orderable list for the simple timeline. Aside from the fact that goocanvas is based on gobject, there isn't much support for dynamically resizing and repositioning objects. I need to build up dynamic containers first, and then make the individual container items reorderable.

There is no consistent property interface between the stock canvas items. The majority support x, y, width, and height. But a few do not, namely Text, Circle, Polyline, and Path. Fortunately, I will only be using the Text object, so I think I can safely assume that all the widgets involved support those properties.

Relationship with Advanced Timeline

The simple timeline is both a simplification and an abstraction of the core objects. The advanced timeline maintains a closer mapping between PiTiVi core and the user interface features. This means that many things which can be done in the advanced timeline will not be visible in the simple timeline. So what happens if the user makes a change in the advanced timeline, then switches back to the simple timeline, and makes a change there? Will some data be lost in the translation? Should the simple timeline attempt to keep track of changes to the advanced view and prevent them from being lost?

The two main issues I can see are:
1) in the advaned view, sources may overlap each other
2) in the advanced view, source and effect properties will vary over time

Both of these can't be represented in the simple view. So what happens if you spend an hour carefully tuning volume settings in the advanced view, then switch to the simple view and move the volume slider? Does it amplify the whole curve? Does it flatten the curve? Do we even allow the volume slider to move after changing settings in the advanced view?

I have a blog again!

2007-06-10T10:47:00.000-07:00

Yes, I have a blog again. Why? Because I have a reason for one. This blog is going to document my progress on the google summer-of-code project. I have been making contributions the PiTiVi Video Editor, a very interesting project which has a lot of promise.

You can already see some of my progress here:
http://apart-dev.blogspot.com/