Calendar Update Part II - After The Release

Following up on my promise to detail some of my future plans. But first, a word about that nasty memory leak.

Memory leak in pycairo or pango/cairo.


I am 80% confident that there's a bug in the python bindings shipped on ubuntu 10.10. I developed a minimal example, (full source here), which just draws some text repeatedly in a window. The relevant bit of this example is the following function:

def draw_text(self, cr, text, x, y, width):
        pcr = pangocairo.CairoContext(cr)
        lyt = pcr.create_layout()
        lyt.set_text(text)
        lyt.set_width(pango.units_from_double(width))
        lyt.set_wrap(pango.WRAP_WORD_CHAR)
        cr.move_to(x, y)
        pcr.show_layout(lyt)
        return lyt.get_pixel_size()[1]

This little script will om nom nom your memory -- over 100 mb of memory after just a few minutes, and I think we can all agree that even if this it's not the most efficient example, its memory usage should be nearly constant. I haven't yet opened a bug, because I still don't know which component(s) contain the actual leak. At least feel better that it's probably not something I'm doing wrong, which gives me the confidence to move forward with the release.

To mitigate against the leak, I threw in an optimization that ensures only one pangocairo context and one text layout is created, per expose event, per widget. This seems like a reasonable thing to do in general, so I will probably leave this optimization in even after the bug is fixed. Unfortunately, all this does is slow down the leak. The leak is still there, which means you still wouldn't want to leave the calendar application running indefinitely.

In passing I'd like to say that the idea that a pango layout needs to be associated to a particular context bothers me a lot. I wonder if I can get away with re-using layouts with different pangocairo contexts.

A better grammar

The current parser for recurrence expressions could use some serious help. The parser for the existing grammar is in the "proof-of-concept" stage of development. It's intuitive for simple things, but you will notice that parenthesis are needed to get correct parsing of more complicated examples. This is due to unresolved ambiguities in the grammar.

Most of the ambiguity could likely be resolved with associativity and precedence declarations, but when you're developing a new notation it's difficult to know in advance what the intiutive associativity is. Basically I didn't bother trying to get it right at all, and rely on parenthesis as a crutch. I recognize that this neither intuitive nor natural. Even I am surprised at how wrong the unparenthesized parse can be.

I am also aware that this approach is horribly anglo-centric. You would have to write separate parsers for  every language you wanted to support, and it's a good bet that language constructs don't always map onto each other so neatly.

I am also playing around with graphical approaches to the same problem, but to be perfectly honest they've all sucked so far. I might do a separate posting on one of them just to show you how god-awful it is.

A mouse-driven way to add recurrence exceptions.

When dragging a recurring event, the current behaviour is to shift the entire recurrence pattern. This is a deliberate choice on my part: it's a novel feature that I want to highlight it. Unfortunately, many times time what you actually want to do is shift just the occurrence you've selected, and occasionally you want to move all the events after the selected occurrence.

Note, you can do this now: however, you have to go through the text interface. Even I find this quite tedious. The solution most obvious to me is to introduce control and meta modifier keys, but I have found that other users do not share my enthusiasm for mouse modifiers. In usability parlance, we say that   modifiers are less discoverable than more explicit features.

Another idea would be to add some tool bar buttons that silently add modify the recurrence rules of the selected event. This would be fairly easy to do, but I have a couple of misgivings. The first is that the toolbar is getting crowded already. The second is that it won't be obvious what those buttons do at first. Clicking on the "except this occurrence" button will not produce an immediately visible change, only modify the behavior of future interaction. This would be especially be confusing if you hit the button by accident.

A more radical idea would be to place special "handles" on the clip itself. That way dragging from a handle could have a different meaning from dragging the clip itself. I am leaning towards this approach, but it is not without drawbacks of its own. For example, the handles compete for space with the event text. They will necessarily be small, which could hurt usability in a tablet.

Integration with other calendars

Syncing with Google Calendar and / or EDS

importing from, and exporting to, .ics.

This work ought to be pretty straight-forward given how the code is organized, but I haven't really looked in detail at the APIs / File formats I'd be working with. If you're interested in helping out with that, you know where to find me ;)

Event alarms

This might not be necessary if I can use something like EDS as the default back-end. But we'll see. Depending on how hard it is to map the concepts in my application onto the APIs exposed by EDS, it might be easier to roll my own implementation.

Calendar Update

What a crazy couple of months it's been. I have been back in San Francisco since late February, but not found the motivation or time to post until now. Among other things, I've been to racing school, and I've taken up drumming as a new hobby. I also attended the meego conference here, where I was showing off my QML GES demo (another post about this is still in progress).

I thought I'd take some time to write about the work I've done on my desktop calendar application over the past several months. This will stop me from pestering my friends about it, which will make them a lot happier.

• Major cosmetic face-lift.
• Added an infinite-scrolling month view.
• Added support for all-day events, which are displayed in a pane along the top row.
• Added support for recurring events via a natural-ish language parser which supports english-like recurrence expressions such as "every two days from now until october", or "the 3rd wednesday of every month except this wednesday"
• You can easily shift the entire recurrence pattern by dragging a single occurrence with the mouse
• Possible to select the same block of time across multiple days.
• Added support for editable event text.
• Removed dependency on goocanvas. Since all the drawing code is custom, and the entire widget is re-painted on every expose, there really is no point in using goocanvas.

I am considering making an initial release, but would like to address the following issues:

• I need to come up with a name. As my music teacher once told me, "Name your children." Anything is better than cal.py.
• Adopt some system for installation, most likely distutils.
• Currently the calendar widget leaks about 100kb of memory for every expose event. After a few minutes of scrolling and dragging it's pushing 100mb. It's obviously a bug, and I've narrowed it down to the functions I wrote to simplify drawing text. Disabling these functions reduces the memory consumption to a much more reasonable 15-ish mb, Either I am doing something memory inefficient in python, or I've found a leak in pango / cairo. I would very much like to understand that better.
• I recently added all-day events. You can create an event as an all-day event, or you can drag an existing event to the all-day area. But you can't drag an all-day event back into the timed area.

I did some "usability testing" on a hapless friend of mine, and the results were encouraging but showed there was a great deal of room for improvement. Unfortunately, my select-then-add paradigm of creating new events is confusing for those used to applications like google calendar and evolution. In evolution I find the click-to-create behaviour frustrating. Google calendar, on the other hand, seems to get it right.

After the release
This post is getting long enough already. I've also got some ideas for subsequent improvements, which I will summarize here. TTFN

• A better grammar
• A mouse-driven way to add recurrence exceptions.
• Integration with other calendars
• Event alarms

Testing Interactive Code

The preceding work on the Render and Project Settings dialogs in PiTiVi has led to some fairly complicated UI logic, and we haven't really got a good way to test it.

Imagine that we want to test the following trivial python UI

import gtk
import gobject
import random

def b_clicked_cb(button):
    clicked = True
    button.props.label = "Ouch!"

w = gtk.Window()
v = gtk.VBox()
b = gtk.Button("Click me")
b.connect("clicked", b_clicked_cb)
v.pack_start(b)
w.add(v)
w.show_all()

gtk.main()

We want to check that the button label is correctly changed after being clicked. Because of control inversion, we obviously can't code in a direct, sequential style. At the very least we must install a timeout or idle function into the main loop.

This is a naive approach:

def test_case():
    b.activate()
    assert b.props.label == "Ouch!"
    return False

gobject.timeout_add(1000, test_case)
gtk.main()

It doesn't work because of timing issues. The button's label won't change immedately after activate() is called, so you get the following error -- despite the fact that the the label clearly changed.

Traceback (most recent call last):
  File "figure_1.py", line 19, in test_case
    assert b.props.label == "Ouch!"
AssertionError

The obvious solution is to split the callback in twain:

def test_case():
    b.activate()
    gobject.timeout_add(1000, finish_test)
    return False

def finish_test():
    assert b.props.label == "Ouch!"
    gtk.main_quit()
    return False

gobject.timeout_add(1000, test_case)
gtk.main()

It's good enough for this trivial example, but it should be clear that as the complexity of the UI increases you'll end up with yet another maze of callbacks.

Last night I experimented with applying python generator functions to this problem. The idea is to (ab)use the yield keyword as a way of passing control between the test and the mainloop. It's similar to concepts presented in this paper, which describes how continuations can be used to solve the problem of control inversion in web programming.

Quick summary of python generator functions: the yield keyword in python is a limited form of either lazy evaluation or continuations, depending on your point of view. Usually we think of generators as sequences. For example this generator can be thought of as the sequence of positive integers from 1 to 10

def ints():
   i = 1
   while i <= 10:
      yield i
      i += 1

I can treat it as sequence and take its sum, or iterate over it:

print sum(ints())
squares = [x ** 2 for x in ints()]

What actually happens when squares is called is that an iterator is created and returned to the caller. Yield indicates to the interpreter that squares() is a generator, and that the state should be saved so it can be re-entered later. The fact that the state of the function is saved inside iterator allows us to think of the iterator as a continuation (generators are not as powerful as continuations). With this in mind, we re-write our test function:

def test_case():
    b.activate()
    yield
    assert b.props.label == "Ouch!"

Here the yield keyword is going to provide the same control flow boundary that splitting our code into separate functions did earlier. Now all we need is a bit of code to consume values from this iterator until it is exhausted. We do this using timeouts -- as we did in earlier examples -- which allows the mainloop to continue running in between the two halves of our test case.

def run_test_case(iterator):
    print "Tick"
    try:
        iterator.next()
    except StopIteration:
        print "Test Case Finished Successfully"
        gtk.main_quit()
        return False
    except Exception, e:
        print "An error occured"
        gtk.main_quit()
        return False
    return True

gobject.timeout_add(1000, run_test_case, test_case())
gtk.main()

This is already an improvement, but not yet flexible enough. Suppose we add a new widget:

def c_clicked_cb(button):
    def set_label_async():
        c.props.label = "Ouch!"

    gobject.timeout_add(random.randint(500, 5000),
        set_label_async)
c = gtk.Button("Async Operation")
c.connect("clicked", c_clicked_cb)
c.show()
v.pack_start(c)

Now we have a problem: we have no idea when the action triggered by clicking the second button will complete. Simply waiting for one second will not always work. At the very least we should be able to override the default sleep value. But it would be better still if we could wait until the label value itself is changed. That way if the action takes only a short time, we don't have to wait, while if the action takes longer than expected, the test can still finish successfully. In other words, we shouldn't just assume that after each step in the test we straight on to the next one. Let's factor out portion of the loop that does the scheduling:

class Sleep(object):

    def __init__(self, timeout=1000):
        self.timeout = timeout

    def schedule(self, iterator):
        gobject.timeout_add(self.timeout, run_test_case, iterator)

def run_test_case(iterator):
    print "Tick"
    try:
        scheduler = iterator.next()

    except StopIteration:
        print "Test Case Finished Successfully"
        gtk.main_quit()
        return False

    except Exception, e:
        print "An error occured"
        gtk.main_quit()
        return False

    scheduler.schedule(iterator)
    return False

Now we can easily customize the timeout for the second button specifying a Sleep scheduler with a different timeout.

def test_case():
    b.activate()
    yield Sleep()
    assert b.props.label == "Ouch!"

    c.activate()
    yield Sleep(6000)
    assert c.props.label == "Ouch!"

Actually we can go one better. We don't have to rely on timeouts for scheduling at all. For example, we can easily define a scheduler that will wait for a signal to fire:

class WaitForSignal(object):

    def __init__(self, obj, signame):
        self.obj = obj
        self.signame = signame
        self.iterator = None
        self.sigid = None

    def schedule(self, iterator):
        self.sigid = self.obj.connect(self.signame, self._handler)
        self.iterator = iterator

    def _handler(self, *args):
        run_test_case(self.iterator)
        self.obj.disconnect(self.sigid)

Adopting this is just a one line change to test_case():

def test_case():
    b.activate()
    yield Sleep()
    assert b.props.label == "Ouch!"

    c.activate()
    yield WaitForSignal(c, "notify::label")
    assert c.props.label == "Ouch!"

And we don't have to touch run_test_case() at all. I think this idea could be expanded into a framework for testing event-driven code. True, I would want much better error reporting. But with just that, it would be pretty straight-forward to cover every part of PiTiVi's interface except the timeline (for the Timeline, I need is the ability to synthesize raw input). If necessary, I can include other types of scheduling scenarios, such as waiting for a file or socket access. And, because I can work with the widgets directly, it's possible to verify conditions that would be impossible to check for under Dogtail or LDTP (which are both limited to what AT-SPI exposes, and run the test from a separate process). Full Source

And the winner is...

The overwhelming majority went for version B. I changed the layout a bit, and a cleaned up version is what's in my branch now. Thanks, everyone.

Round 4

So it's down to two competing ideas. At this point my brain is shot and I can't really decide between them. So I've decided to put them up here and see what people think. The best way to compare the two is right-click the images into separate tabs and switch between them a few times. Take a good hard look at each one.

What I'm looking for here is clarity of ideas, not cosmetic considerations like alignment and balance. That is, which of these dialogs is easier to understand?

A

B

Now that you've done that, does anything seem to be missing from either one that would make it an improvement over the other?

Round 3

09:38 < bemasc> emdash: So, the true parameters here are the storage height,
storage width, PAR, and DAR.
09:39 < bemasc> that's 4 values in total, but only 3 degrees of freedom.
09:40 < bemasc> Personally, I would expose all 4, and update PAR after each change to DAR (and vice versa)

How about this, then?

Starting to look a bit better? I kept the notion of 'Display resolution' mainly as a source of feedback, but it occurs to me now that perhaps little rectangle that represents the display aspect ratio would be a better idea. Unfortunately that would mean increasing the size of the dialog again.

I'm actually not so sure that DAR and PAR should update each other, as I played around with it and it's somewhat frustrating. I think most of the time you're either targeting a specific format (for example, 480i) in which case you'd just choose a preset, or you'll be working with square pixels.

I think i need to add a 'constrain aspect ratio' check-button to make scaling a bit easier.

Project Settings Progress

Re-worked it a bit this afternoon. But i'm still not happy with it. I think it might be too confusing to expose pixel-aspect-ratio to the user (hell, it's confusing enough for the programmer).

Another idea: present display width / height, optionally allowing the user to constrain to an aspect ratio. Separately present content (storage) width / height, and finally show the pixel-aspect-ratio as a read-only label, as a form of feedback. In other words, PAR would be computed working backward from display and content resolutions.

Mockup: The diagram on the bottom shows how updates
would propagate between widgets

PiTiVi Dialogs

I am (finally) making progress on the two main dialogs in pitivi: "Project Settings" and "Render". I'll just get right to the point: Screenshots!

Render dialog showing video tab

Render dialog showing general tab

Project settings showing video tab

I spent most of my time today on project settings. I'm still not quite happy with it, as the current design is, well, huge. I think I could make it a little narrower and shorter. But that's more-or-less what I'm after.

The properties in the video tab are inter-dependent. One feature I wanted was for the widgets to ripple-update like a spreadsheet. To make this easier I wrote a class that manages a graph of widgets that you give it. Each widget can have an update function, and when the user changes one of the widgets, the the update functions are called starting from that widget in breadth-first order. During the traversal, updates are ignored to prevent infinite loops of signal emissions. This means I get my ripple updating behavior, but I only have to write a single, naive update function for each widget without worrying about keeping other widget values in sync.

Tomorrow I will implement presets, make sure the dialogs remember their state, and then test the hell out of them.

Side Project

I started working in a calendar app. Right now you can think of it mainly as a prototype, but it's already got some interesting features. It's implemented in python, and while it relies on goocanvas at the moment, the calendar view is actually one big canvas item and could easily be ported to a stand-alone widget.

Interface

At first glance it looks like the week view used in most timelines. However, the view is an infinite scrolling area. There is no horizontal scroll bar, as this is a ranged widget. You move in time by clicking and dragging on the day heading. If you flick the heading, the calendar will continute to 'coast', slowing down gradually until it stops (or immediately if click on it).

I've found that I rarely look at the month view in a calendar application, so I sought to eliminate it. My calendars tend to have a lot of daily repeating events, making the month view seem cluttered. I mainly use it for skipping forward or backward to a specific week without loosing context, or to see a group of days that span a weekend.

With infinite scrolling I found I really don't need the month view. I don't loose context skipping around because of the smoothness of the animation. Moreover, it's no trouble to position the calendar so that Friday is in the middle of the window (the coloration on the heading serves as a reference). Expanding the window will reveal more days, not make the days larger. If you want to look two weeks ahead, you can -- even on an eee 901. I do plan to add a zoom feature, however.

Selections

Another feature I've added as a reaction against current calendars: When you click and drag on the calendar itself, it doesn't create a new event automatically. I've always found this to be annoying. When I make a selection, I am usually in some kind of tentative state of mind. Moreover, in thunderbird and evolution, it's possible to accidentally create events that are too small to be selected, forcing you to go into another view to delete them. Again, really annoying. I don't want the application taking some action based on my half-baked ideas. I much prefer it to wait until I'm sure of myself.

Undo / Redo

This applet is the first time I've implemented the Command Pattern in a program, and I'm curious to see how it pans out. The idea is that rather than employing low-level tricks to capture state for undo, you explicitly code the do and undo procedure for each action that your application supports. So far, New, delete, move, resize, and select can all be undone.

I think that selection is too often left out as an undo action. Doubly annoying because selections are ephemeral, disappearing the minute you focus on something else. Often I select things without thinking. I've put a lot of work into creating selections, only to have them evaporate due to an errant click.

On the implementation side, I distinguish between MenuCommands, from which menu and toolbar items can be created automatically, and MouseCommands (invoked repeatedly with new mouse coordinates before being committed to the UndoStack in their final state). Both types support a class-methods indicating whether they are currently available. This allows toolbar state to be managed, and for the controller to distinguish between one of several commands it might support.

One thing I found is that with this approach, the Controller becomes a general-purpose object which mostly creates and commits commands to an UndoStack in response to low-level events. The controller implements the logic for detecting drag, click, double-click detection. The commands implement more concrete actions, such as moving an event in the calendar based on given mouse coordinates. The Controller only needs to be specialized to add new input events. For example, flicking the calendar is currently implemented with a custom controller.

In the future I plan to make navigation undoable. I'm debating whether navigation commands should be posted to the main UndoStack, or into a separate stack that would be exposed as "Forward / Back".  I'll have to try and see.

Development Status

If I follow this out to its conclusion, it would need at least the following features before I'd call it "Done".

• Ability to edit the names of events
• tricky because the whole thing is done in raw cairo
• Some system for handling repeating events
• convenience navigation buttons, such as "Today", "Last {Week, Month, Year}", "Next {Week, Month, Year}" etc.
• Alarm notification system, and ability to handle alarms
• Integration with google calendar (probably using libgdata)
• some cosmetic improvements

The implementation of the alarm notification system worries me most. I have only a vague idea of how I would implement that, and it would need to be absolutely reliable before I'd consider using this program myself. I do have some interesting ideas for how to create alarms and repeating events in the UI, however.

Shopping

Coming home Saturday night I was delighted to see that "The Co-Operative" grocery store nearest my house was still open. I went in to buy some things for next morning's breakfast.

There were two choices for bacon: one claiming Britain as its origin, and another from "the family farms of the Netherlands". At this point I must confess that my first reaction was "Eek! Why would I want Dutch bacon? I'm in England! The British bacon is for me." It seems I have begun to adopt a British point of view after only a few short weeks. That is until I looked more closely at the package and read that it consisted of "87% pork."

I was perplexed. How in he world can 13% of a slice of bacon consist of something other than pork? Some minutes passed, during which time I surveyed the contents of the package. Perhaps closer inspection would reveal a portion of the of the slice where some miscreant had inserted turkey or beef. At least that would be a reasonable explanation. Alas, I found no signs of obvious adulteration.

Unsettled, I chose the Dutch variety.

Scene from a small boat punting up the Cam (the river in Cambridge).

Some More Pics From Spain, Continued...

I ate at this really neat cafe in Figueres. 

The bathroom was entirely stainless steel, and used car door handles.

Here I am, trying to be artistic

There's a lot of really good street art in Figueres...

...as well as the more typical variety.

More scenes from Figueres.

Waiting for the train back to Barcelona.

In front of the Dali museum, with a dopey grin.

It's worth the trip. Thanks for the tip, Kat!

A closeup of the the dots on the exterior of the

building. Each one appears to be slightly

different.

Some More Pics From Spain

I've been out of Spain for a couple of months, but I still have plenty of photos I never posted. Truth be told I found it a bit painful to manually upload them. I don't know why it never occurred to me to use Picasa until just now. Enjoy.

Tarragona

Scenes from a public concert in Montjuic

A warm waffle, covered in chocolate sauce and sweetened condensed milk. Purchased from the above stand.

So, I'm in Cambridge, UK now. I've been meaning to post up for days but haven't quite found the time. I'm going to try to get some longer posts out in the next couple of weeks, but for now I'll settle for something short and simple.

Quote of the Week

The other day I was walking through Midsummer Common, which is a big green field near the center of town that normally contains quite a few cows. Instead there was some kind of bicycle race event and the place was packed with people and cycles. I smelled the smell of good, greasy food -- like the kind you get at a county fair or a race track -- so I sauntered on in. As I was munching on a tasty roast pork sandwich (with stuffing and apple sauce) I overheard the following snippet of conversation:

Boy (in a whiny English kid voice): So, can I mum, can I?
Parent: No Roger, you've been an absolute muppet today!

That's the first time I have ever heard the word "muppet" used this way. I wasn't sure whether it was a fluke or not, but while house-hunting today, my would-be roomate used the term in the exact same way to describe someone who forgets to lock up the house when they're the last to leave.

Nerd Section


Some time ago I got distracted by a silly idea that just wouldn't leave me alone. It kept gnawing at me to follow it up, so eventually I gave in and started hacking on it. Have a look at the following code. I'll give you three guesses to tell me what language it's written in.

     DEF (id(outer) FUNCTION(id(x)
       FUNCTION(id(ignored) VAL(x))))

     DEF (id(one_ret) APPLY(VAL(outer) _(1)))
     DEF (id(two_ret) APPLY(VAL(outer) _(2)))

     PRINT (APPLY (VAL(one_ret) _()))
     PRINT (APPLY (VAL(two_ret) _()))

     DEF (id(count_backward) FUNCTION(id(x)

           IF(NOT(VAL(x))
              PRINT (_(done))
              BLOCK(
                PRINT (VAL(x))
                APPLY(VAL(count_backward) MINUS(VAL(x) I(1))
)))))

No, it's not LISP. The parenthesis go on the outside of the expression.

No, it's not python. Notice how there's no commas separating arguments.

Did I hear someone say haskell? Now you're just being silly.

It's actually C. Through abuse of variadic functions and preprocessor macros, I implemented a functional, dynamic language directly in C. For the curious, you can see the full example at my git repository, http://github.com/emdash/goof/blob/no_commas/simple.c:

I did it for fun, but it does work. Bonus points for explaining how it works in a comment.

Returning Home

I'm a bad blogger, I know. I didn't even post before leaving spain :( In the last couple of weeks I managed to do some traveling around Catalonia, seeing Taragona and Figueres. Dali Museum is really quite awesome in Figueres. Taragona has roman ruins and a good beach. Have pictures, but not on this computer. Will try and post them soon.

I'm currently in Boston visiting some friends on my way back home. My friends live between Harvard and MIT. Walked around the campus and then my friend took me down town to see boston common, and the harbor. We also passed by the grave yard where Paul Revere, John Hancock, and the victims of the Boston Massacre are buried. John Hancock's tomb stone is the largest of the bunch. While walking through downtown Boston we ran into Richard Stallman. That's all I've got for the moment.

Today I am thinking about busing to New York city and back for a bagel, a hot dog, and a pizza. But we'll see. Jet lag and my friend's cat have combined to give me a bit of a jumpstart on the day. I'm looking at the schedules right now.

Getting Ready for GUADEC

The last few days have been a welcome reprieve from the oppressive summer heat. Thank god for that. Of course it cools down just before my departure to the Netherlands for GUADEC.

Going to GUADEC next week. There's going to be a big PiTiVi hackfest involving the majority of the current PiTiVi contributors. I'm really looking forward to some serious progress.

Also hoping to give a lightning talk about some hair-brained ideas I have for UI development.
Last week, my friend Aaron came to visit and we rented scooters and rode around the city. It was hot, but fun. Enjoy the photos.

This is us up at Parc Guell.

Did you ever think you'd see Aaron on a scooter?