Posterous theme by Cory Watilo

Accessible HTML (and CSS) Toolbar

Have you ever caught yourself creating a toolbar element for your new web application like the following, and hooking some javascript to set a selected class on the selected tool?

Well, as most of the things related to software development, there are countless ways of solving the same problem. The difference between these solutions varies from using the wrong tools to code elegance, readability and maintainability (which are debatable as well, but that's a whole different conversation).

When it comes to the HTML+CSS+Javascript trio, I always push to have them as separate and decoupled as possible. I mean, no Javascript messing around with styling, no semantic-less CSS classes in the middle of the markup simply for easing the styling process. I acknowledge that sometimes (sometimes!) we need to be a little flexible and give in to a few hacks, using our best judgement, of course.

Going back to the toolbar, we can describe its behavior as having a single element selected from a list of elements. And guess what... We've just described HTML input radio's (aka radio button) behavior! So how about not reinventing the wheel and using what you already have in hands?

The following markup is all we need, in HTML, to have our toolbar setup.

If you switch to the result tab, you may say "It doesn't look like a toolbar to me...". And you are right! It doesn't look like a toolbar because there is no style attached to it. Lets then add some styling. With the current markup, because the labels are tied to the radio buttons by their IDs, the labels are going to be our buttons. So these are the steps we'll need to take in order to make this toolbar actually look like so:

  1. Hide the input radio elements;
  2. Hide the text within the labels;
  3. Style the labels as buttons.

Voila! Here is your accessible toolbar! Accessible because screen readers can still read the tools names. Try playing around with it. Details about the stylesheet are inlined as comments in the CSS tab below.

If you need toggle buttons, like bold or italic in a text editor application, for example, the HTML checkboxes are what you need. And you can use the same approach to style them appropriately.

So the bottom line is: before creating your own bunch-of-divs element and hooking some Javascript to define its behavior, take a look at the standard HTML elements. And if they match the behavior, but don't look like what you expect, style it!

Cheers!

Front-end HTTP Server for Play 2.0 applications with WebSocket Support

It has been almost a month now that I have deployed a Play 2.0 application on Linode. I intend to explain it in details on a near future, but for now let's simply say that it enables one to search the Freenode Scala channel logs. The channel is logged and indexed on the server side, and on the client side the query is sent through a WebSocket in Javascript. It's good to remember that the app is written in Scala and can be found on my github.

Usually, when deploying an web app, reverse proxies are used to distribute load from incoming requests, cache content, or hide internal apps. In my case, I needed one just because, by the time I put it live, I was using a beta version of Play, which didn't allow me to change its default listening port (9000). I was looking the latest documentation and appears that this doesn't hold true anymore, but a month ago I just wanted the user to be able to access it through port 80.

First I looked for nginx, which has a lot of buzz going on around it. Unfortunately, based on what I read on "the Internets" seems that make it work with WebSockets support isn't that easy, involving the compilation/configuration of some 3rd party module.

That's when I directed my attention to another option: HAProxy. Although it may be hard to believe that it works when you first visit their website, I read good things about it and its performance. I think the cliché about not judging a book by its cover fits quite well here.

Before putting it in production I tried it on a development box, which had the latest version of Ubuntu and a recent version of HAProxy. After getting some examples and reading the manual, I ended up with the following config file, that turned up to work well (it probably still might be improved).

In production I preferred to adopt a LTS version of Ubuntu (10.04), but the HAProxy proxy version available on the repo wasn't that new. However, luckily for me, compiling and installing it is as simple as getting the sources, the build-essential package and running:

make TARGET=linux26 CPU=i686
mv haproxy /usr/sbin/haproxy

Feel free to take a look on it working. It might take a while to load some searches with a lot of hits, because it's generating snippets for all matched lines. There are also some bugs with the generated JSON on search results, which then end up breaking the Javascript side. I will be addressing that soon.

How was Node Knockout?!

Some of you might have heard that this past weekend Node Knockout (nko) was happening. If you don't know what's that, think of it as Rails Rumble for node.js. Still didn't get? It's a 48h competition where a team should create an app (of any kind) using node.js. Myself and another group of friends were participating and it was a great experience. We decided to share here some of it.

Our team was named Latest Build, with 3 members (including me) working from ThoughtWorks office in Porto Alegre, and another one coding from San Francisco. The competition started on friday (Aug 26th), 9pm our time, and finished on the following Sunday at the same time. In total, each of us probably slept less than 6h between those 48 ones.

We created a web app called twetalk. It allows attendees and presenters of a talk to interact with each other through tweets. Accessing our app, a conference organizar can create the schedule of it (together with the respective talks) and then everyone is able to send tweets to any of the talks. Those tweets are stored, so you are able to check what is going on (or went) whenever you want. With that...

...presenters can:
  • have feedback on their presentations
  • notify public about related news (share slides, etc)

...attendees can:
  • provide feedback
  • discuss talk contents and subject
  • know other attendees
  • look for presentation news

A better description of it can be found of the about page.
Screen_shot_2011-08-31_at_4
Please, don't forget to check it out and vote :)

For creating it we used express. Looks like it is the most famous web development framework so far, and to be quite honest, it worked quite well. For twitter authentication we went with everyauth (worked greatly as well) and for using the twitter API (searching tweets and updating user status), we went with node-twitter.

Under the hood we went my MongoDB and mongoose ORM. Mongoose is still very raw and we had different opinions about it: some liked, others not that much. But it was Mongo that surprised me on a negative way. It was the first time I used it and There are a few basic features that we needed that were not there (although there are open tickets for them). Those are support for triggers and search inside nested arrays of a document.

A lot of the communication between server and client side was done using WebSockets with now (mainly the back and forth of tweets). It worked quite well, except inside TW's network (again, take a look on 'The bad' part).

Client side was a lot of jQuery, Twitter's Bootstrap, and fullcalendar, a jQuery plugin for creating calendars that look like gcal.

There were three options for deploy: a Joyent VPS, Heroku cedar stack, and a Linode VPS. Looking for a solution for the WebSockets problem, we ended up trying all the three options.

Heroku quickly became one less. It didn't had the power of the VPS solutions and it doesn't support WebSockets (at least on the free account), which is pretty bad. Also, in order to create a Mongo instance, we would need to give our credit card informations (I was not trusting our software that much :P ).

Joyent had a Solaris machine. Although I like ZFS a lot and really trust Solaris, we didn't felt as confortable with it as with a Ubuntu server with apt-get that we put on Linode. That's why we chose the second.

Some tools that were very helpful: nodemon, for automatic redeploy on development; npm for package managing;MongoHub, a MongoDB Mac OS GUI.

My personal opinion: the node framework and surrounding packages are still very raw, but I'm quite impressed on how everything worked in the end. Also, we used node.js correctly, with little server-side processing and a lot of IO (twitter, url minimizing apis, db, etc). Even though there are probably dozens of bugs in production, I believe that the app (using happy paths only) can scale quite nicely by using the node.js asynchronicity.

On the end, we forgot to expose a few features to the user. When the user creates a talk on the server, we created a minimized URL for it. If someone tweets anything with that URL, it will be automatically added to the twetalks of that talk (not needing it to be tweeted inside our app).

The good 
  • Red Bull really works, specially the big cans;
  • Although available frameworks for node are in a very infant state, they are plenty. We always had like two or three options to consider;
  • Learned a lot of stuff on those 48h;
  • A lot of people on the event's IRC channel willing to help;
  • Our friend in SF broadcasting the nko opening event from Joyent's office through Skype.

The bad
  • Our office network was not supporting the WebSockets version we were using :( That was a really sad part, because we spent 4h debugging, talking with a lot of people, etc, etc, to then switch to a 3G network and see everything working just fine;
  • The immaturity of available libs and frameworks;
  • I'm feeling tired and our girlfriends are mad with us :P

The ugly
  • Go horse doesn't scale well on the long term. As development went on, we were spending more and more time to manually tests for our changes. I'm not saying that if I started back, I would do everything using TDD, but that at some point, for sure, the time spent doing unit tests would be smaller than the time needed to do the manual tests. I really don't think that's the case for a 48h spike, though;
  • After the deadline, we found a problem where invalid data inserted in the DB could break the page of a specific conference. Since we could not do any more deploys, but we are allowed to do any kind of sysadmin activity (other than manually changing code), we created a parallel process (in node, as well) to fix that bad data. Every 5 seconds it looks for wrong entries and update them with valid values on those fields :P

If you have any questions, suggestions, interest in helping future development, please let us know. As soon as the judging phase is over, we are planning to release the source code.

Tips for your Distributed Project Inception or Meeting

My current assignment is done together with a distributed team, in a different country. From time to time we have new projects starting with them. Usually we travel to their site, but once in a while the whole process is done remotely.

Normally, our inceptions take from 1 to 2 weeks. At first, a Distributed Project Inception may look like any other normal meeting we often do (it's worth to remember that we also do remote stand-ups daily), but I assure you it will become a nightmare if you don't make some proper decisions at the beginning of it.

Based on my previous experiences here are some ideas you might consider when doing your Inception, or even any other kind of meeting where part of the team is not physically present.

 

Use proper tools

A distributed inception or meeting in general can go all the way down if you can't understand what is going on in the other side. To properly understand what the guys on the other side are doing/speaking, you need good tools. I'm not only talking about a clear voice reception, which is mandatory, but also video, which is just equally important. It's way better to understand what is happening on the other side when you can see the actions of the rest of the team.

That's where tools like Webex and Vidyo are great. If you don't have access to those tools, consider buying a premium account for Skype.

Furthermore, don't forget about mic, speakers and camera. There is no point having the best tool if the microphone or speakers you are using are crappy as hell. Nowadays We've been using a ClearOne Chat 60, which is working successfully with a team of about 10 people. That stands for cameras as well. Avoid using that cheap one that comes with your old laptop. Instead, use a good USB one with enough resolution for the task.

By the way, another very important thing to keep in mind: if you have a camera, use it! Don't hide on the corners trying to avoid it. Speak on the front of it, so that the rest of the team can see you. That's very important for a good communication.

 

Have a plan B

Bandwidth can be a problem. Depending on where you are, power supply can be problematic as well. There are several things that can go wrong during a remote meeting. And like Murphy once said, "anything that can go wrong will go wrong". Along with one of the tools cited before (or any other of your preference), think about having a open phone line muted and with the volume down. If the first one fails you can quickly switch to the second option until the service is reestablished.

Don't forget time zones and daylight saving time

It might happen on the first time and you really don't want to miss the meeting just because you got confused with the clock. That's easily avoidable by using a calendar application that support multiple time zones.

Laptops/cellphones/etc are not allowed

During meetings, it is deadly easy to lose focus. This is even worst when the rest of your team is across the globe. As soon as get minimally lost in a discussion, there is a big change that your next move is pull your cellphone and check your e-mails. That will soon lead you to twitter, facebook,... you name it.

This rule applies to any kind of nearby toy as well. It's better to just leave them outside the meeting room and come back to them after your meeting is done.


Let the remote side read what is on the screen

It happened several times with me, where the other side is driving a shared screen and the person driving the computer keeps moving the browser/spreadsheet/whatever up and down, changing windows, etc, and the screen on your side never get's updated with what they are actually seeing. All you see is a messy image with a lot of pieces of several applications :(

Every time that a new point is about to be discussed and part of a document needs to be read out loud, let the remote side (the one that is not driving the screen) read it. That will ensure that they also can see what is being displayed. Better yet to not have a fixed guy, but doing that interactively, choosing another person each time.

Time boxed tasks

No matter how good is your tooling, it's very tiring to have remote meetings. You spend a lot of effort to understand everything that is going on: different voices, accents, subjects, etc. So, don't think you have all the time in the world to discuss that specific subject, and even if you do, that's not productive. The more you take to end a conversation, more tired will be the participants and, likely, worst will be the final result.

That's why you should define a time box to discuss a specific topic. For instance, having a frame of 5 to 10 minutes to discuss a story and the the tasks related to it. That will make everybody focus on the current goal, leaving aside any kind of distractions.

Use big screens/projectors

Com'on, they are getting cheaper day-by-day. Even if your team has only 3 people, why put everybody staring at a 17" monitor? That will harm your health and make you burnout faster.

If you have different stuff to be shared, like the video for the other team and also some application screen, look for a second monitor, or as many you might need to nicely display all the shared information.

Remember the rest of the team

Although they are not sitting at your side, they are there and they do want to hear what you have to say. Avoid speaking only with your local team or have side conversations. Don't forget that muting your microphone can be very impolite too. It's weird to see mouths moving on the TV, but no words coming out from the speakers. That opens room for various assumptions on what is being talked on the other side.

If there are too many people talking at the same time, consider using a token on each side. Only the person with the token is allowed to speak and after finished, he/she should select the next one to have the token.

Feel free to request someone to repeat of what he/she just said. It's important to have everybody on the same page and to remember that they are part of your team, and not as "the guys on the other side". You'll probably need them to have a successful project.

That's what I can remember so far.

What about you? Anything that has worked well with your team? Let us know!

Isolating the flapping tests

A while ago, near a branch-cut to move all the new features that we have been developing for a month to the pre-production environment, our regression test suite became red and, for our bad luck, there was a different test failing each time.  Moreover, for every breakage, someone spent some time looking for the issues just to discover that everything was working as expected.

What was the reason? Non-deterministic tests. In fact, this proverb shows the pain point of flapping tests:

A liar's house is on fire and no one believes him. - Turkish proverb

When the regression suite starts 'lying' to your team due to non-determinism, you should spend some time to fix them. Otherwise, everyone will start disbelieving on its results and no one will stop whatever they are doing to fix the application.

After those bad days dealing with non-deterministic tests, we started looking for a solution and we found a Martin Fowler's article that helped us a lot: Eradicating non-determinism in tests.

That post encouraged us to create a quarantine environment and we are now reaping the benefits!

As you can see on the image below, we started fixing the functional tests isolated from the main suite in the quarantine environment. Doing so, we avoid breaking the test suite and we keep our work flowing. On it, each line, is one of the functional tests of our suite. While that, the first two tests have been fixed. The third and fourth still under analysis to find the root cause of the indeterminism.

 

Quarantine2

As you can see, there are other benefits. We can see the failure rate of a given test, and we can run them more often than it would normally run. With the rate, we can then know if the test is really flapping or it's a bad implemented one.

How did we implement the quarantine?

As we are using RSpec version 1, which hasn't tags (available on version 2), we did a monkey patch to run the tests in quarantine even when they were marked as pending. For that, we have a file where we keep reference to all the flapping tests, and the framework is able to separate those tests and run them. In this way, they won't run in the regular environment.

 

Are you seeing a non-deterministic behavior in your test suite? How did you stop it or plan to do that? Share your experience and let's stop them!

A Taza Cheat Sheet with best practices and conventions

At my project, here at work, we (developers) usually play not only the Dev role, but sometimes Quality Analyst (QA) as well.

One of the tools we usually use for browser acceptance tests automation is Taza. Taza itself is a very powerful framework and very open, allowing you to abstract/write your tests using different approaches. That's a good thing, but unfortunately, when developers don't agree on a code convention, a huge mess can grow in the codebase.

A few days ago we had several discussions on how could we write better and cleaner functional test code and the output of that we arranged on the attached Cheat Sheet.

We hope you like it. Let us know if you find any mistake or have any suggestions.
Click here to download:
taza_cheat_sheet.pdf (410 KB)
(download)

ScalaTest: writing expressive unit tests on the JVM

Recently, the presentation I submitted to Agile Brazil 2001, about writing unit tests for the JVM with Scala, was accepted and then I decided to write here some posts about/related to it in order to get more feedback before the actual presentation.

There are different libraries with that purpose in Scala, like specsyspecs² and ScalaCheck, but ScalaTest has been my favorite so far. Like you can find on their website, "ScalaTest™ is an open-source test framework for the Java Platform designed to increase your productivity by letting you write fewer lines of test code that more clearly reveal your intent". In my opinion, the best part of it is that ScalaTest is a very flexible framework, letting you use other testing frameworks, ones that you may already know, like JUnit or jMock, or even use different styles of testing available on it.

Despite the name, ScalaTest doesn't aim only Scala, but also Java. Since in the end of the day Scala code becomes .class files as well, you can write tests with it for both languages. Once there are several legacy system with Java, and even more projects coming and using it, that becomes a great thing.

But before digging into ScalaTest, let's just take a quick look on Scala (I will not cover the details of the language, since there are several other articles on the Internet that talk about that). Check these two code fragments written in both languages:

First, a classic example: the definition of a Person class with name and age attributes. While you need more than 20 lines to write that in Java, Scala requires only one, since we don't need to write setters and getters that will be generate to us by the compiler (yes, we are not accessing the attribute with person.age, that's a generated getter). The second piece shows a small method that checks if a String contains a uppercase character. In that you can see the use of closures in Scala. It also shows that Scala syntax is a little different from Java. Those two examples are good to show the power of the Scala and how, with more features, syntax and paradigms, you can write less yet more expressive code.

Let's jump back to ScalaTest now, and the first thing you might ask is: "how I write tests with it?" The response here is kind of funny: that's up to you! Like I wrote before, ScalaTest provides several traits that you can mix together into you code, giving you or your team the option to chose the combination that feels most productive. On this post I will start with tree basic traits: FunSuiteShouldMatchers and BeforeAndAfterEach. The first is one of the most basic type of test Suite, where each test is represented by a function, similar behavior to a method annotated with JUnit's @Test. The second trait adds a DSL for tests with the should keyword (That will be more evident on the examples). The third one is very simple: it just adds methods for set up and tear down of the tests, like @Before and @After.

But first, we need some code to test. That's why we have our ultra super mega powerful TicTacToe game written in Java. Nothing very special on it. We have a couple of exceptions to address unwanted use of the game, a Enum to define he possible states of a square (X, when a X is marked; O, when a O is marked, and _ when nothing is marked on that square). The game itself with its rules go in the TicTacToe class (simple as possible):

 

Back to the FunSuite... as a initial step, the simplest test suite we could write is something like this:

Not many secrets on that. We created a class that extends FunSuite and uses the test keyword. Each test receives a string that describes the test ("testing hello world"), and the test block. That's the output you get when you run it on a command line:
Screen_shot_2011-05-13_at_12
Let's write a real test now, still keeping it simple. Once a game starts no one should have won it, right? :P 

On this test, we are doing a very basic test over our game and using the simplest form of assertion, that is done through the assert keyword. However, this example doesn't show the true power of using Scala on unit tests. 

Although the test looks very simple to understand, it could be even more descriptive. If you are familiar with RSpec you have already seen the should keyword, which looks great when unit testing. What if we could do that on Scala too? The truth is that we can. Adding the ShouldMatchers trait to your test callows objects to be asserted through a should method. Let's move on now and modify a little that previous test:

We are using two matchers on it now. The first one (should not be) receives a symbol ('finished) that corresponds to a method on the object under test. ScalaTest will them will invoke that method, expecting a Boolean to be returned. But what you can see is that the test itself looks just like a normal english phrase. The second matcher (should be) used is a little simpler, just asserts that the return of ttt.winner is the same as the object inside parenthesis.

For other matchers, take a look on the trait's documentation. There are several ones for checking Strings, Ints, collections, and other basic types. If you don't like should that much, you can also use must. Just use the MustMatchers trait instead.

At my github account, you can find the full test suite I made for the TicTacToe game using the FunSuite. Other interesting thing you will see there is this other test:

With that, we are initially marking the square at position (0,0) and then trying to mark it again. With the should matcher, we are checking that this move is generating the expected exception. Once more you can see that the test looks just like plain english.

That's it for now. Next time I will write about the use of JUnit together with ScalaTest, since it's one of the more common test framework used Java. I hope that may help your switch to Scala, allowing you to start to write more expressive unit tests on the JVM.

Cheers.

Good Ol' IRC

This might not be a surprise to some of you, but IRC is comming back (and stronger)! Well... at least among the software development community.

I remember the days when regular non-techie people stayed late at night joining channels, chatting with their friends and using the Internets with an IRC client. When AIM, ICQ and other instant messaging applications came into existance, people started using them. This migration happened mostly because of the difference between purpose and need:

  • Instant messaging focus more on one-to-one chat, having a list of friends, user friendly interface - your grandma can use it! - and the notion of presence (available, away, typing indication...).
  • IRC is focused in group chatting, having  subject-specific channels where people would identify with that group and join... And it is not so regular user friendly.

Around the same period, internet forums were being created and techie people started using these new information sharing places to discuss about technologies, programming languages and other geeky things.

Given that you have to wait until someone sees your question and post a response, I can see a decline in the usage of forums in favor of using the good ol' IRC, which provides a much faster question/answer interaction.

Lately I've been doing some experiments with Node.js and I'm making extensively use of IRC to help and get helped. I am amazed of how the community is making such a good use of this tool.

I decided to write this post when I got my second real-time bug fix, check it out:
(A little bit of context, I'm using LearnBoost's Stylus in my Writeboard project and tjholowaychuk is one of its maintainers.)

Do you feel the same?

Scala 2.9.0 Parallel Collections

Scala 2.9.0 just got out. Among the new features are the parallel collections. With them, by calling the new par method, every collection may be converted into a corresponding parallel collection:

scala> val l = List(1,2,3).par
//l: scala.collection.parallel.immutable.ParSeq[Int] = ParVector(1, 2, 3)

I just downloaded the new version and wrote some quick test:

Scala's Benchmark trait allows you to time the execution of the code block of the run method, which can be ran N times with the runBenchmark method. Although the first discrepant result on both benchmarks, the numbers look good. That's the output I got (time is in milliseconds):
Time to run Normal Collection: List(110, 129, 93, 110, 88, 127, 113, 112, 118, 130)
Time to run Parallel Collection: List(191, 71, 82, 78, 68, 78, 68, 90, 77, 68)
Screen_shot_2011-05-12_at_6

With this release, together with the announcement of Typesafe, Scala is becoming not only a great general language, but a even better option for concurrent, parallel, and distributed computing.

ps.: it was ran in my personal machine, with two processors. Imagine that on a server with several cores :P