It Even Works!

Software Engineering | Highly Available Services | Security | Pretty much anything

Analyzing a live Ruby process

So if you are a C guy you are probably use something along the lines of


$ gdb attach (pgrep myprocess)

..

(gdb) bt

to check what is going on when you need to debug something. If what you want to look into is written in an interpreted language (say Ruby) gdb will take you too deep inside (to the actual interpreter’s program state). So what do you do?

In Ruby you can see your current stack via:


p caller.join(\"\n")

Note that caller returns an array of strings each one containing filename, line number and method of a given point in the stack (hence join(“\n”) to print it out nicely). So you could use this along with a signal handler (say SIGUSR1) as a window to look into your running scripts:

Signal.trap("SIGUSR1") {
  puts caller.join("\n")
}

This can come in handy when your codebse is somehow big. I am also sure there might be better methods to do this and I haven’t looked into what facilities are being build into future versions of Ruby to enhance the debugging experience. So I’ll probably add more about this in the future :)

Small fix, huge savings in typing

I love automating stuff. To get some work done I need to access services that require password authentication. In order to do that, my scripts need to talk to gnome-keyring (which exposes the org.freedesktop.Secret.Service D-BUS API) to get my credentials. The usual dance (from a Ruby script) goes along the lines of:

require 'rubygems'
require 'dbus'

bus = DBus::SessionBus.instance
service = bus.service("org.freedesktop.secrets")
secret = service.object("/org/freedesktop/secrets")
secret.introspect
secret.default_iface = "org.freedesktop.Secret.Service"
search = []
unlocked, locked = secret.SearchItems(search)

Strangely, this stopped working after I upgraded to Fedora 17 (beta). Digging around with d-feet I stummbled across the /org/freedesktop/secrets object which wasn’t exposing the org.freedesktop.Secret.Service interface as it did before. So I cloned gnome-keyring’s repo and started looking around. I found out that before the interface definitions were distributed as XML companion files but that has been changed to inline strings. In doing so, a small bug was introduced by which the Secret Service component referred to the wrong interface. So I filed a bug along with a 1-liner patch and Stef was kind enough to review it right away.

I don’t have to type my password anymore and my scripts can work without me. Hopefully this’ll get picked up before Fedora 17 leaves beta status.

Beautiful code

While reading the SPYD book I learned about this very nice patch for Firefox to use warmed up pipes first:

https://bugzilla.mozilla.org/show_bug.cgi?id=624739

The core part is:

+            // Keep The idle connection list sorted with the connections that
+            // have moved the largest data pipelines at the front because these
+            // connections have the largest cwnds on the server.
+
+            // The linear search is ok here because the number of idleconns
+            // in a single entry is generally limited to a small number (i.e. 6)
+
+            PRInt32 idx;
+            for (idx = 0; idx < ent->mIdleConns.Length(); idx++) {
+                nsHttpConnection *idleConn = ent->mIdleConns[idx];
+                if (idleConn->MaxBytesRead() < conn->MaxBytesRead())
+                    break;
+            }
+
             NS_ADDREF(conn);
-            ent->mIdleConns.AppendElement(conn);
+            ent->mIdleConns.InsertElementAt(idx, conn);</pre>

Small details matter!!!

New munin packages

Bernie has been complaining for a while that we are getting some errors in the logs when running munin-node from cron:


not a reference at /usr/share/perl5/Munin/Master/Utils.pm line 901

Instead of actually debugging the problem (and since we are running a beta version – 2.0.beta4) I just decided to update to the next available version (2.0.beta7). So, if you are using Ubuntu here are some packages for your to test:

http://www.itevenworks.net/~rgs/ubuntu/

Now go test!

Sugar Gtk3 Hackfest @ Prague: day 3

Today was the last day of the Gtk3 Hackfest and more code was committed:

  • loads of small gtk3 fixes by Daniel and Benjamin
  • Because gdk_property_change is not introspectable, I commited a small work around for it
  • I ported the last bits of gtk2 in the Toolkit to gtk/gdk 3 plus added a wrapper for rsvg while it gets support for introspection
  • Daniel got a basic version of Browse based on WebKit running
  • Walter did more progress porting Abacus
  • Marco and Benjamin did more progress on the palettes
  • Simon fixed lots of introspections bits in the XSMP code (X Session Management Protocol) and also other Gtk-isms here and there

All in all, I think we’ve set the foundations for a smooth coexistence of gtk2 and gtk3 activities though we still have a long road ahead.  Stay tuned!

Update: a ported version of Abacus:

The code is here.

Sugar Gtk3 Hackfest @ Prague: day 2

Yesterday was day 2 of the Gtk3/Introspection hackfest and we got loads done:

  • Tomeu committed a patch to make the XSMP stuff in the Toolkit introspectable
  • Marco and Benjamin kept working on porting the palettes to gtk3 (they are almost there and should be committing today)
  • Simon and I polished the sugar-toolkit-gtk3 branch
  • Daniel made sugar-activity (the launcher script) Gtk agnostic so we it can load either gtk2 or gtk3 activities. Part of this was swallowing sugar/activity/main.py into sugar-activity. Though Marco later pointed some shortcomings of this approach. Also, the logger and dispatcher from sugar-base were moved into the Toolkit
  • Daniel also hacked on Browse to drop hulahop and add WebKit  (this is for demo-ing purposes, not a full working WebKit enabled browser yet)
  • Walter started porting Abacus to Gtk3
  • Bert worked on Etoys’ collaboration stack

Update: here is what you need if you’d like to test all of this code: http://wiki.sugarlabs.org/go/Features/GTK3/Development

Sugar Gtk3 Hackfest @ Prague: day 1

Yesterday we had day 1 of the Sugar Gtk3 hackfest at the brmlab and some interesting discussions and tasks were triggered:

  • Marco and Benjamin went over porting our palettes to Gtk3. Of course after a heated discussion Marco couldn’t sleep and he followed up on the mailing list with a potential solution to the dilemma.
  • I rebased the gtk3 patches for Sugar Toolkit on top of master.
  • Daniel committed support for styling the ProgressBar in gtk3 (sugar-artwork).

We adjourned for further discussion of whether we should branch of the gtk3 version of the Toolkit or host it as new project entirely. We concluded that we’ll:

  • name the new module sugar3
  • the tarball sugar-toolkit-gtk3-version
  • the 0.94 branch will be used for gtk2 only releases (bugfixes only)
  • will commit the gtk3 stuff to master and follow from there (git mv sugar sugar3)
  • encourage distros to ship both as separate packages
  • see if we can drop the custom mime handling stuff in sugar-base (in favour of something equivalent in glib2 perhaps?)
  • maybe move the logger and dispatch code in sugar-base to sugar-toolkit

Porting sugar-toolkit to Gtk3

If you are following Sugar development (and of course you are!) you know that we are almost done removing hippo in our path towards Gtk3 and the new (introspected) world. The nice thing is that there are many other less invasive improvements that can be achieved before we get there. One of them, extensively explained and documented by the never at rest Daniel Drake, is the ability to ship a secondary gtk3-enabled Sugar Toolkit for all the cool kids out there that are looking forward to write nice and fresh activities. This also means you get access to all the other introspection goodies out there, most notably access to the latest WebKit bindings.

As a warm up for the Gtk3 hackfest starting tomorrow, I wanted to go over what we last did at the Desktop Summit along with Benjamin Berg to obtain a gtk3-enabled Sugar Toolkit. So here is what was done to port sugar-toolkit to Gtk3:

Converting the Toolkit to Gtk3

  • run pygi-convert.sh provided by the pygi community to automatize migration to gtk3 as much as possible
  • the script isn’t perfect so you’ll have to add a couple of imports (GObject and Gdk) here and there by hand
  • set_geometry_hints is broken, comment it out
  • use Gtk.Menu instead of sugarext.Menu
  • GtkAlignment doesn’t take named parameters anymore
  • use get_window () instead of self.window
  • vborder/hborder in Gtk.Notebook note recognized
  • pack_start() doesn’t take implicit params anymore
  • Gtk.Label#size_request() new type of return value
  • Gtk.icon_size_lookup and size-request have new API
  • sugarext: avoid using sugarext.GsmClientXSMP

In any case, my ported branch is here: http://cgit.collabora.com/git/user/rgs/sugar-toolkit/log/?h=gtk3

Renaming  the ported sugar-toolkit (s/sugar/sugar1/g)

Because we intend to ship two versions of the Toolkit, for gtk2 and gtk3, we need different namespaces. So we go ahead and rename ’sugar’ to ’sugar1′ with an incantation along the lines of:

find -iname "*.py" | xargs sed -si 's/sugar/sugar1/g'

Ship it!

Before installing the new Toolkit you might wanna a save a copy of the original (gtk2) Toolkit so it doesn’t get overwritten. This can be done via:


cd /path/to/sugar-jhbuild/install/lib/python2.7/site-packages/
mv sugar sugar-gtk2

And then to install the new Toolkit:


cd /path/to/sugar-jhbuild/source/sugar-toolkit
make install

and finally do some renaming so each Toolkit can be find in the proper location:

cd /path/to/sugar-jhbuild/install/lib/python2.7/site-packages/
# make the installed Toolkit the alternate (new) one
mv sugar sugar1
# restore the original Toolkit
mv sugar-gtk2 sugar

Finally, a small hack to sugar-activity so it imports the correct toolkit
by replacing /path/to/sugar-jhbuild/source/sugar/bin/sugar-activity:

import sugar

with:


import os

if os.path.exists("GTK3"):
     from sugar1.activity import main
else:
     from sugar.activity import main

Now you should be able to seamlessly run both gtk2 and gtk3 activities!

Embrace gtk3!

So here is a quick gtk3 activity (CrazyTurtle) i wrote while thinking and exploring how would
porting TurtleArt to gtk3 go. You can find the source code: http://cgit.collabora.com/git/user/rgs/CrazyTurtle.activity/.

Run it!

Lets give our new gtk3 activity a try:

A couple of obvious things to be fixed:

  • no theming (go benzea, go!)
  • palettes are broken
  • testing needed

The Future

Other cool stuff that might come on top of this is a new version of Browse (or Surf?) that is fully gtk3/webkit enabled along with many new and shiny HTML5 apps. There is also a whole world of opportunities around gtk3. One of the things that made me wonder about cool stuff we might explore is Alex Larsson's new broadway backend for Gdk. I can't come up with any immediate use from within Sugar but it is good to know what possibilities are available out there.

vborder/hborder in Gtk.Notebook note recognized

Hippo removal status: almost done

Last weekend I was at Sugar Camp Paris. Besides meeting up with great people who are pushing forward really cool Sugar deployments, Daniel Drake, Simon Schampijer and myself had some time to keep working on the patch set that removes Hippo from Sugar and opens up the path towards the much desired Gtk3 migration.

What we achieved:

  • lots of code clean up here and there, Daniel reworked the CanvasIcon class and also the Intro window.
  • all views are now working, though the code is far from optimal state and there are some rendering bugs (there are some mysteries around the Grid class used for icon allocations too)
  • the Journal and all of its view are looking pretty good!

We are looking forward to have a new sprint this weekend and then maybe send this off for review. Hopefully, we’ll be able to land this before the upcoming Prague Gtk3 & Introspection Hackfest.

Hippo-free Sugar and Sugar Toolkit repos are here:

http://git.sugarlabs.org/~erikos/sugar/no-hippo

http://git.sugarlabs.org/~erikos/sugar/no-hippo

Gnome Contacts will rock

I’ve been following up the development of Gnome Contacts which uses libfolks for aggregating contacts and is meant to come out with Gnome 3.2 and things are looking pretty good!

Yesterday Sjoerd said that it would be very cool to be able to make calls, both regular VoIP over XMPP calls or using SIP to reach PSTN lines, from Gnome Contacts. So I started hacking on it and came up with the following:

Calls using Gnome Contacts, libfolks and Telepathy

As you can see, UI needs some work so hopefully Design Ninjas might be able to give some feedback before UI freeze.

Code is here while I keep working on this until it can be merged. Stay tuned!

Update: you can find RPMs for Fedora 15 here.