It's Over!

Well, it's not really over, but yesterday, the 18^th, was the last day of coding for Google Summer of Code students. Service clients are 90% done as of this writing, and it shouldn't be hard to get the other 10% done in a day or two, plus some bugfixes. But deadline or no deadline, service clients are a sugary-sweet feature that XMMS2 really needs, and I will be hacking on them for a long time to come.

SoC certainly solidified a few lessons about the real world for me. Most importantly, in my humble opinion, is that money changes things. When you're coding for money, you're coding under a deadline, and it's incredibly easy to fall behind. The result/value split code that service clients depended on was supposed to be completed and merged before SoC even began. It's only now that SoC is over that the result/value split is "done" and it's still unmerged. The API for my project changed half a dozen times before I wrote a line of its code. And a myriad of personal issues took days and days of quality hacking time away from me. Lucky for me my mentor(s) were great in helping me sort out issues with the design and code, and reestablishing goals to fit the program into the summer. Thank you, nano and theefer!

A sure bonus is I now have a jumpstart on understanding the result/value split code and all that it brings. Since it is a pervasive change to both the client library and server-side, there's no getting anywhere without knowing how to work with it. Plus, as a member of the XMMS2 team, I'll be there after SoC to help merge, document, and answer questions.

Now with the finish line in sight, the thought of the upcoming mergefest and eventually, users using service clients has me more motivated than ever before. And I'm filled with a sense of great pride that I stuck it out through the summer and created something I feel is worth being proud of. But for now, I'm going to play a quick round of DDR with my little sister and get back to work.

Next on the Agenda...

Well, the client-side of things is mostly done for now, so it's time I turned my attention to the remaining items. The server side of things needs to be updated/fixed and the service client manager (SCM) needs updating. The server is fairly straightforward, but I have some concerns about the SCM.

• Does the SCM record clients in its configuration when they are run? Probably not. How would it record the executable path, for example? We should find a way to make this more automatic.




• Provide methods to read/write SCM configuration.


• service_client --register #registers its configuration using above methods


• The above command can be called by waf during build to eliminate user intervention. The clients are already registered and ready for use upon installation!




• Can we rename the SCM's shutdown function to "kill" (and perhaps redefine its functionality) or something that can't be confused with xmmsc_service_shutdown?


• Can we change the default value of the "monitor" option (to check the configuration directory for updates) to true? This seems to cooperate much better with user expectations.


• Can we get rid of the "timeout" option (to wait before killing a service client after a shutdown request is received)? If we rename the method to "kill" its intentions will be a lot more obvious--take down the client now.

Visions of Grandeur, Llamas, and API

I've been really bad about updating this blog, I just noticed. So here's some fresh, tasty meat to dangle in front of your snapping jaws. The past several weeks were met with real life issues and coding issues, but not a lot of real code. But very recently I started working on the new API, and it's looking exciting!

Here's the vision so far. A service client (SC) creates a new service, stored in a service_t. The SC then can add methods by giving it a name, description, and arguments, each with a name and type. There is a variadic version of this which lets one add a method at a time, and a more primitive version to add a method, then one argument at a time. This makes it easy for C programmers and binding authors alike.

Here's what the data structures look like:

service_t:
 char *name;
 char *desc;
 uint32_t major;
 uint32_t minor;
 x_list_t *methods;

 uint32_t ref;

service_method_t
 char *name;
 char *desc;
 void *udata;
 xmmsc_service_notifier_t func;
 xmmsc_user_data_free_func_t ufree;
 xmms_value_type_t rettype;
 xmms_value_t *args;

The data is held in value_ts whenever possible because these are the new universal type being introduced with the result/value split code. A value_t can contain any of several basic types and can be serialized and deserialized automatically. Its representation and usage is effectively the same on the client and server. Because a service_method_t must contain function pointers and user data, it has to be a struct. (How could you send a function pointer over IPC?)

The next step is for the SC to call service_register and register its service with the server. Internally, service_register handles the conversion from service_t and service_method_t to representative value_ts, then sends them off to the server.

A client can request any information about the service from the server using service_describe. Since the data is already conveniently encapsulated in value_t form, it is just shipped off to the client. The client can then see everything except the actual function pointers, udata, and refcount (why would it want these anyway?).

Said client can use this data to figure out, say, the version of the service is within its own supported range. So the client queries a method. Let's make this more interesting with an example. We'll make a client LlamaGUI that implements support for counting llamas using the LlamaCount service.

void llamagui_llama_count_get (connection_t *xc)
{
  value_t *farms = list_new("llama_farm_a", "llama_farm_b");
  value_t *args = dict_new("farmlist", farms);

  res = service_query(xc, "LlamaCount", "count_get", args);
  service_notifier_set(xc, res, llamagui_llama_count_callback);
}

This request is sent to the server using the provided connection xc. The server simply looks up the "LlamaCount" service and shoves the request right along to the sc. Back in the sc, the request is checked for well-formedness. Are all the required arguments there? Are they the correct types? If not, a value_t containing an error is sent back through the server to the requesting client. If so, the notifier method is called, processing the request:

value_t *llamacount_get (connection_t *xc, service_method_t *method, value_t *args, void *udata)
{
  int count = 0;
  value_t *farmlist;

  farmlist = value_dict_get (args, "farmlist");
  for(; value_list_valid(farmlist); value_list_next(farmlist)) {
    count += llama_farm_count (value_int_get (farmlist));
  }

  return value_new_int(count);
}

This starts the return trip through the server and back to the requesting client. There is one more check performed to see that the return type indeed matches the type given at registration. A service method may always send back a value_t containing an error. If the types do not match, however, an error will be returned instead of the original value_t.

void llamagui_llama_count_callback (value_t *ret)
{
  if (value_type(ret) == VALUE_TYPE_ERROR) {
    llamagui_error_set("Oh no! A llamacaust occurred. Error %s", value_error_get(ret));
    return;
  }

  llamagui_llama_count_set(value_int_get(ret));
}

The LlamaGUI user gets her llama count right next to the XMMS2 playback controls and playlist of llama-friendly songs. Everyone is happy.

It Begins

Summer of Code 2008 officially kicked off on May 26. Within two days, I had about 1500 lines of C written for the result/value split needed for service clients to have a clean API. I started out by writing a patch that was just enough to split xmmsc_result_t into xmmsc_result_t and xmmsc_result_value_t. Then I got the suggestion to create common implementations of all the types used by XMMS2's IPC and encapsulate them in an xmmsc_value_t. That accounted for about 1400 of the 1500 lines of code. :)

As it turned out, all that work wasn't necessary and only the original patch was needed. Fellow XMMS2ers theefer (Sébastien Cevey) and nesciens (Erik Massop) took over from where the patch left off, completely separating the two structures and even converting the CLI code to use the new value_t. So now, service clients will be able to pass arguments and return values around conveniently as xmmsc_value_ts.

Now the big question in my mind is how do I merge this beast? Somehow, I need to get service client code on top of the result/value split code. Do I simply rebase the code and resolve every conflict I can see as I go? Do I squash the service client code down to a few commits? What about authorship in both cases? One thing is certain; it's going to be very hard to visualize everything that needs changing to use the new value_t. But I guess this is just another lesson in how developers do things in the real world and why small, incremental commits are so useful in preventing this situation.

So I suppose there is really only one option: to rebase and pray that it's not impossible to fix. But that's the biggest obstacle in the way of getting service clients into the next release. Anything for the t-shirt!

Service Clients Redux

It's that time again. People are walking around in flip-flops, getting tan(ner), and licking ice cream from cones. It's summer (or it's close enough, anyway), and this summer I'll be getting my tan from the computer monitor and my ice cream from the freezer. It'll be a basement-dweller's summer for me as I'm coding on XMMS2 in this year's Google Summer of Code.

I'll be working on service clients, a continuation of last year's unmerged work. The concept is to provide so-called "service" clients which perform a service (surprise!) that can be re-used by other clients so client authors aren't reinventing wheels all the time. It's just another way XMMS2 eases music player development to the point where making your own interface is a weekend project instead of a thesis project.

What Service Clients Are

The client-server nature of XMMS2 is a constant issue when trying to add new features or fix bugs. What was trivial to accomplish in a monolithic music player is not so trivial in XMMS2, and vice-versa. Service clients are a way to provide the same whipped-cream-and-cherry features you'd find in a monolithic client, but in a reusable, modular, and extensible form. It's cleaner than adding new stuff straight to the daemon, but still fairly simple and straightforward to use.

Service clients work by registering themselves and their callable methods with the XMMS2 daemon. Regular clients (or even service clients, as they're regular clients, too) request a service client from the daemon and call its methods appropriately and get back data. It's that simple. Album art lookup, tag lookup, last.fm submissions, and FreeDB lookup are all easily written as service clients and their functionality is very easily included in any client by use of a few methods. Services are even browseable so, with a little user intervention, a new service client could be used from a client that has no idea how to use the service, at least in theory.

What This SoC Will Bring

Service clients were part of last year's Summer of Code program, completed by student Ning "zeegeek" Shi under the direction of mentor Sébastien "theefer" Cevey. The API works and there are a couple service clients floating around, but zeegeek's codebase was never merged with the main XMMS2 tree. The code needs a full review and some questions about remaining oddities answered. A little bitrot has accumulated since last summer, too. My job will be to clean up the code and get it merged, first.

Later I'll also be writing the first language bindings in Ruby. This, too will be followed by a merge. Too often, Summer of Code projects are completed but remain unmerged for months, creating lots of work later. This summer, I'm including merges as part of the project goals. Once the service client API and Ruby bindings are completed and merged, it's time to take service clients' virginity. It won't be just a framework anymore, but I'll implement some useful clients, starting with a port of xmms2-scrobbler. And of course, this'll be followed by a merge as well.

Then more cool clients will follow. The next generation of collections, (Collections 2.0) also being implemented this summer, will probably implement the collections parser and a more flexible party shuffle as service clients. This is an exciting collaboration opportunity. Other possibilities include a medialib-updater styled service, taglib client, FreeDB/musicbrainz lookup, and album art lookup. I'm outlining way more service clients than I can handle this summer in the hopes that I can pick out an important and doable few and get those few done and merged by the end of SoC. They'll also be indispensable means of clearing up remaining kinks and bugs in the code while establishing some good example code and helping me write documentation.

Working on service clients this summer is exciting and I'm overjoyed that this is my first SoC. This is an excellent project with an excellent team during an excellent summer, so I've little doubt that all this year's projects will be successful. I'm hyped, and you should be, too! Stay tuned for when I finally settle down with a six-pack and start exercising my fingers.