New year, new project

I mentioned earlier that I am a bit frustrated with SUnit as it currently is. This is because every time I go look at it or have to modify it slightly for something, I end up thinking that it is in need of refactoring.

Since this refactoring is of the large scale variety, modifying the current SUnit would possibly break existing test cases which work. Therefore, I decided SUnit needs to be rewritten.

And well, at least a mock draft of that has just begun. The name of this project was chosen because I think SUnit has potential for much more than just unit tests. I think the current naming convention (unit tests, tests cases, test results) does us a disservice because it is as if it implicitly prohibited any use of SUnit other than to write unit tests, which is silly.

What we do with tests, regardless of whether they are unit tests, or validations, or benchmark testing, are all instances of the same thing: assessments on various objects such as code, domain objects, or performance characteristics.

Therefore this new stuff is called Assessments.

We'll see how it goes.

Optimizing C compilers, hmmm...

My first structured programming language was Pascal. Back in the day I became quite proficient at squeezing the size of the compiled executable by means of careful development. I also spent considerable time looking at what worked and what didn't in terms of execution speed. Further exposure to assembler only emphasized such learning.

For example, I still remember to this day that back then I got ahold of a library to manage fancy input fields called GICSTR. The original had about 4600 bytes of compiled code. By the time I was done with it, I had a rewritten one which required about 2200 bytes of compiled code. Both did exactly the same things.

Then came Smalltalk and for many years that knowledge stayed dormant, at least as far as things like CPU instruction scheduling was concerned. While I like optimization in Smalltalk as well, it's not the same thing as counting CPU cycles.

So nowadays I write C code, and I am somewhat disappointed to find out that even after all the talk about SSA, optimizing compilers and so on I can still go ahead, tweak C source code with assembler in mind, and lo and behold the VM runs faster. For example, it looks like I may have just squeezed another 5-10% out of the nfib benchmark by further massaging of the small integer addition/subtraction primitives.

But besides the speedups which are always nice, what bothers me is that I should not have to care about the manipulations I am doing. What is this optimizing compiler business if I still have to do manual work? Sigh...

In the mean time, I guess I will end up swapping the old Pascal/assembler optimization library in so I can modernize it with more current C and x86 assumptions. More speed is on the way.

Revolution, evolution, and convolution

Many times I've heard, in the context of software development, that a computer revolution is still needed, and how this somehow means we need to find out what is going to be the next technological development that will change everything.

The way I see it, what we are looking for when we ask for the next set of shiny, glass programming language beads is an excuse to start over from scratch and to hide our inability to deal with our current obligations using current technology under the rug.

Back in the day, Java was the biggest thing. A-ha. Today, it's Ruby or Python or whatever. And tomorrow there shall be many others as well. At the same time, we still use C and we do not even dare say it's old news.

More importantly, however: have we satisfactorily addressed the underlying issues of software engineering, or do we still do the following?

• Software module dependency is handled poorly, e.g.: Vista.
• Buggy software is shipped regardless, e.g.: Leopard.
• A high percentage of software projects still fails miserably.
• Projects deemed successful do not do what the customers wanted in the first place.
  
• And, more than ever, it is possible to call oneself a developer without even becoming familiar with things like modular arithmetic.

So if this is the state of affairs, how can we claim to be prepared to make sound judgments about our past, current and future technology?

I still do not find proper reasons to justify our apparent need to start over, or to invent the future if you will, because I think we cannot do our job properly today. What kind of next big thing can we invent if we have not even mastered what we already have? How can we claim the new stuff will solve our problems if we have not yet clearly understood what those problems are?

Therefore, my friend, when I hear we need a new approach to everything and therefore all our problems will magically go away, my answer is no. Domain problems will remain just as hard regardless of whether we use Smalltalk, assembler or Cobol to address them. That is the point here: we are getting confused because we think the domain issue is a technology issue.

But why do we do that? Because usually domain problems can be solved with rudimentary technological tools. We are not afraid of the hammer, of course not. What we are afraid of is building a house using a hammer. Since that is scary then we claim we need the next big thing, perhaps an extreme hammer, to build houses properly.

Of course we use the extreme hammer with its bright red or military camouflage handle as some sort of talisman so we can fool ourselves into feeling confident enough to go do what we have to, but we do not say that.

So if we need to build a house, do we study civil engineering or architecture? Of course not, because we are scared of that too.

More than a revolution, what we need is a good dose of evolution to go past our current state of convolution.

Nothing to do, so...

So I thought I'd go back to one of my lovely past times for a while, and found a couple things I didn't know. How about this?

The sum of the binomial coefficients (n, j), for 0 <= j <= k, is the same as the sum of the binomial coefficients (n-j-1, k-j) * 2^j, for 0 <= j <= k.

Pick any place inside Pascal's triangle, and the consider the diagonals going up from the place you selected. The sum of all the numbers in both diagonals is a power of two.

Lovely stuff!

More hash book for you

I'm getting good feedback, and this results in more work and more text: the draft just reached 460 pages.

Time

As of late, I've been spending considerable time thinking about the passage of time. The writing is on the wall: there's just too little of it available.

Sometimes when I say things like that I get comments such as "ah but you are young, what do you care about running out of time". But you know what? I want to write books today because they require an amount of energy and dedication that I seem to be capable of spending today, and I do not know if I will have that capability in the future.

Just as my eyes progressively lose their ability to focus on near objects, I feel the effect of the years in other things too. For example, I've noticed that the amount of attention I put into everyday stuff has diminished considerably over time. A friend of mine commented on something that I am becoming strangely familiar with as well: a particular kind of context switch becomes more costly, and stack spills in short term memory cause what appears to be a freeze of the task scheduler. Is that a sign of getting bored and just not paying as much attention to everyday stuff anymore? Is that a sign of being tired? Or is it an early sign of things to come?

In any case, one only has to look at older people to know what's coming. Therefore, one thing is for sure: I am not going to wait until it's too late. The time to do things is now.

Hash book...

I really need to stop working on it, because everytime I touch it what happens is that it grows. Now it is 458 pages, has a new introduction section of 4 pages, and the final chapter now has a bonus exercise section with 7 exercises... including a truly beautiful one regarding binomial coefficients. Sigh... at least, it's RC2 now.

Together with the mentoring course book, I am up to 796 pages --- and since I have 15 or so pages already written for the AGC book, I now realize that I am past the 800 page milestone. Amazing.

And watch out, because this Saturday my sister and I will create the cover for the mentoring book. Ah my friend, I am telling you, it's a work of art.

Happy holidays!

Ah my friend, I've been working on a few little gifts for you in the last few days... fine, I admit they were pet peeves of mine to begin with. But if everything goes through, then at some point in the future you will have:

• 12% faster nfib benchmark,
• up to 40% faster message lookup in some cases,
• faster small integer addition and subtraction,
• faster small integer multiplication, and
• >2.5x faster largeInteger bitAnd: smallInteger when both are positive.

Cheers!

PS: Hmmm... I had further thoughts about method lookup and I think it can be made to be significantly faster than what is stated above. We shall see.

Work, work, work and more work

I read somewhere that when you take care of your job and meet your responsibilities, the reward is more work and more responsibilities. Sigh, so true.

So what does the pending task list look like now? Let's see...

• I am growing tired of the current implementation of SUnit. Each time I extend it or talk about its extensions it turns out there's more stuff to refactor because it gets in the way. Test selector lookup is quite inefficient by design, creating TestCase instances and filling the test selector in is certainly hackish, making subclasses to get specialized behavior is difficult because code is not refactored, etc. I am almost certain it needs to be rewritten from scratch. One of these days, that is exactly what will happen.
• I need to write a little introduction to the hash book, thus making it RC2.
• Further work needs to be done regarding the book covers. The one for the hash book is close to getting done, the one for the mentoring course looks good on paper and now it needs to be flushed out for real.

The amount of work required to do anything of decent quality is just amazing. I feel the need to offload work that is almost done just so that I can just swap out all that stuff and swap in other things that I need to take care of. Hopefully the books will be (really) done soon.

Material from Smalltalks 2007

Ah, my friend, here are some links with material from Smalltalks 2007.

The presentation materials are here.

There are two sets of photos available: this one (taken by me and my sister) and this one (taken by James Foster). There is also a set of photos of Buenos Aires (taken by James) here.

Finally, the video* shown during the conference breaks is here. It includes greetings from many people (thank you!), as well as a number of photos from some Smalltalk projects and research labs that are doing very well.

Enjoy!

PS: you can get the video in other formats here.

*: the video is encoded as an AVI file with DivX for the video and LAME MP3 CBR for the audio, with the usual one second audio preload. Its duration is almost 14 minutes.

Photos from James Foster

James Foster of GemStone fame provides two sets of photos taken during the time he spent in Buenos Aires for the Smalltalks 2007 conference. Enjoy!

Photos of Buenos Aires.

Photos of Smalltalks 2007.

We're still working on the official media package of the conference. Stay tuned for more stuff!

Smalltalks 2007, thanks

Conferences like Smalltalks 2007 do not happen because of a few people. They happen because of everybody. Therefore, I'd like to take some time to express my gratitude.

I'd like to thank the people that took time to come over and join us at the conference. Without them, our meeting would have been meaningless. Thank you for your enthusiasm, your efforts, your collaboration, your energies and your participation, as well as your numerous messages with nothing but kind words for us the organizers.

Then there were the volunteers who helped us with the organization of the conference. There was just way too much stuff to do besides hosting the event, such as the preparation of the bags with goodies for everyone, or manning the registration desk to assist people coming in.

In addition there were the sponsors of the event. Thanks to people from ESUG, Cincom, Instantiations, GemStone, Snoop Consulting, UBA, LIFIA, Mercap, Caesar Systems, InfOil and Seminarios Athena we had everything from the large room with movie screen for the conference to coffee/tea and snacks for the attendants. Thank you so much to every one of you for your support. It really made a huge difference.

And then, well... after all is said and done, it was four of us: Andres Fortier, Hernan Wilkinson, Leandro Caniglia and myself. Thank you for this wonderful experience of organizing a conference with full confidence and trust in each other. Without this kind of collaborative effort, Smalltalks 2007 would simply not have happened.

Cheers to everyone for this success!

Smalltalks 2007, Wednesday

Finally, we had a GLASS workshop on Wednesday. About 20 computers were used to develop a website with very little effort, which was hosted on James Foster's laptop.

So let's see... you set up your repository, and then creating 2-3 classes and writing these most silly of methods gives you a website with photos, CSS, a login window, and content which is rendered on the fly?

I am telling you: editing HTML by hand or even by tools looks silly by comparison. Why bother with that when you can simply use a code browser in your web browser and do as you royally please, knowing that everything you are doing is in this rock solid persistent back end called GemStone? Why bother with anything else? GLASS will make a whole bunch of stuff appear to be obsolete, and hey... I am considering using it myself to replace my flaky FTP/HTTP servers. I want nothing less, and nothing else.

Smalltalks 2007, Tuesday

Tuesday began with a presentation on software development which pointed out several things that cause software projects to fail, such as the vicious circle of buggy code maintained by experts who hack the system further in response to pressure from customers. In addition, to prevent these problems, a number of suggestions were made. For example, there should be a strong culture of testing, and there should be a separate QA team with minimal interaction with developers so that there is no conflict of interest. This lead to more than 80% code coverage, extremely low QA test recidivism, and a logarithmic increase of code size which in turn can be seen as code refactoring working its wonders.

Then there was a presentation about a complex software system that runs both on a desktop and a PDA. It is used to track the activities of a company that recycles clothing and other materials. So for example, the problems that are addressed by the software are the routing of drivers to more than 1200 containers, their activities, the tracking of how much stuff was picked up at each location, and many other logistic issues.

Since the next presentation was mine, I am not going to comment much on it.

After that, the Argentine army came over and talked about their experience with developing applications in Smalltalk. Here are some of the most interesting points.

• Smalltalk is very easy to learn.
  
• There is little ramp up time for new developers. If they only know OOP, then they can contribute within 80 hours. If they know Smalltalk as well, then they are productive within 40 hours.
• Adapting the software to new requirements is easy and can be accomplished without going through major trouble.

What is interesting about these is that if you ask any Smalltalk developer, you will hear the same things. But in this case, there is a study backing up these conclusions against the experience observed in other programming languages. In other words, it is not as biased as a Smalltalk developer would be, and as such it can be seen as external confirmation of what we all feel to be true.

During this presentation, a number of Smalltalk simulation applications were shown. All of them included rich graphical presentation, including for example the overlay of information of military value on top of actual satellite photos.

Then, just before lunch, we saw a presentation on context sensitive applications and physical hypermedia. I wish I had been able to take more pictures, but there was so much information that I just could not make time to use the camera. The first part was about a number of software models that can be used to deal with issues that come up in context sensitive applications (in which software is aware of what the user is doing besides what the user is actively telling the application), and the second part included a demo of a physical hypermedia application running on a PDA which allows users to navigate a city in two completely different ways: by following internet-like links to get more information, and by following physical links which are interpreted in a different way such as e.g.: the user wants to go somewhere, but the traversal is done by the user not the computer, and as such different rules apply. I am really not doing any justice to the presentation by compressing it down to a paragraph, but if I had to file it out here it would take me 20 pages or so.

After lunch, I attended a block of 3 presentations out of a possible 6. First, there was a presentation on how to integrate the SCRUM process to VW's Refactoring Browser in a multi user fashion. It included the following nicest of features: once you tell the tool which item you are working on, then it collects all the changes into a changeset. Then, when you are done, it sprinkles the changeset into the corresponding packages, does all the merge work for you, and even publishes it with the proper version numbers. Everything that can be done automatically is performed for you.

Then, there was a presentation about how to interact with objects in a graphical manner. Mostly, the conclusion is that one should not use a UI builder to build UIs because it causes all sorts of problems and irritations. Instead, if one could attach a set of halos to an open, active window, then one could simply switch the whole thing into edit mode, display anchors and other visual indications to show how widgets are layed out, and then simply change them on the real thing. It was followed by a demo of the actual thing, which was quite impressive.

After that there was a presentation about Squeak 3.10, which included many talking points about the development processes that are guiding the latest release of Squeak. We also saw, besides the release image, an image with a number of things that did not work in previous releases. Hopefully these will see the light of day soon.

And thus we went into the final presentation block of the conference. First we saw a presentation on GLASS given by James Foster of GemStone, including a demo of the GLASS appliance being actively developed right now. What could I say about this other than it was very impressive? The default configuration you get will allow web sites to serve 15 pages per second with a 10 minute timeout in under 4 gigabytes --- for free!

Immediately after that we had a presentation about Morphic 3.0, which includes a UI written independently of the notion of square pixels. In fact, besides having something like 10x less code than the current Morphic 2.0 implementation, it can do arbitrary coordinate transformations on graphics. So therefore if you have a rectangular map, you can project it into the screen via the cylindrical transformation of choice and presto, there you see a map on the screen. Or you could also apply a spherical transformation and then you see an Earth. It also includes other goodies such as subpixel rendering.

And finally, we had a presentation on controlling robots from Squeak via Bluetooth with the intention to play soccer with them. This is not easy to do at all, because you have to interpret images you receive from an assortment of cameras to figure out what to do quickly enough so that your robots can operate as a team. While showing live soccer would have been impossible due to the fact that being able to play currently requires something like 2 days' worth of calibration efforts, we did see the the simulators running on the screen. Amazing.

So this is how the conference ended on Tuesday.

Smalltalks 2007, Monday

Maximo Prieto gave a very nice keynote. In a way, it hurts to see that we have not done much since Simula-67, some 40 years ago. What are we doing today? Objects, classes, messages, single inheritance, polymorphism. All of this existed decades ago. Where's the new stuff? Is there new stuff to begin with or not? Is there, as is there in music, a limit that has been reached past which it is nearly impossible to innovate? Thus the keynote's more than appropriate title, "And now what?".

The next presentation was on how to structure a large financial system as a collection of subsystems which are late bound via the specification of the system composite. This allows catering to different customers with a single code base because what they receive is a different specification of what their software looks like.

The presentation after that was about a large scale refactoring of an oil and gas industry system so that there is a distinction between services provided by the developers so that users can write their own software, and what users actually do to model their business processes.

Before lunch, there was also a presentation on several tools to assist the integration of code changes in large software projects. The big feature of the presentation was a live demo of an Envy-based automatic code integrator, complete with publishing capabilities pre/post condition checking.

During the first afternoon block, I attended two presentations. The first one was Feature Driving Browser, which included the following interesting observation. If you have a lot of tests and a lot of code, sooner or later you will have the problem of tracking down which piece of code is causing some test to fail. If there is a lot of code, then this can be time consuming. What the tool could do for you is to compare the code coverage of tests that pass to that of the tests that fail. The code that is run exclusively by the tests that fail can now be considered as prime suspects.

Then, there was a presentation about how to implement true method delegation (a la Self) in VisualWorks. It was very interesting to realize that delegation is equivalent to a super message send with the exception that the method lookup can begin in any arbitrary class. While the implementation dealt with self/super references properly, some problems such as direct references to instance variables remain.

Then, back into the large room, I attended a very nice introduction to Seaside. While I could not pay a lot of attention to the next one regarding what (to me) appeared to be a macro refactoring of Seaside to eliminate tons of repeated code, the day ended with ST + RIA in which a Flash UI was mounted on top of a traffic simulator written in Smalltalk. Complete with traffic lights, honking and different kinds of vehicles, while the Smalltalk server ran the simulation, the browser displayed the progress in a really nice looking manner.

Then there was dinner, of course. And in this way, Monday came to an end.

Smalltalks 2007 recap

Ok... so after coming back from being very tired, and being awake again, it is time for the organizers to do the rest of the homework.

First of all, here are some stats from the conference. We had almost 300 registrations, and out of those about 200 people attended the conference (perhaps a few more that self registered at the registration desk). One could think this is bad, but it is quite the contrary. See, in Argentina there isn't a strong culture of companies giving employees days off so they can go to conferences. As such, some people could only come to one or maybe two of the conference days. In other cases, we heard of things such as group of 6 developers who wanted to go to the whole conference, but that due to production issues it ended being the case that one developer could go to the GLASS workshop and that was it. It is our hope that in the future this will change and that things will be easier for everybody in Smalltalks 2008.

Oh, by the way... it is pretty much obvious that there will be a Smalltalks 2008, 2009 and so on.

We are currently working on the photos, the materials, the slides, and everything else. These will be released in the next few days. In the mean time, I will be writing short notes about the presentations I attended.

More to come...

Conference so far...

I am sitting at the GemStone workshop, which will start in about 10 minutes. I have not had much of a chance to write about the conference, so let's see what I can do in little time.

First of all, it has been a great experience to be an organizer of a conference. And, in addition to that, the conference presentations have been extremely interesting and very well prepared, and public reaction has been very positive. It may be too early to tell right now --- after all, Smalltalks 2007 has not ended just yet --- but Smalltalks 2008 has already been mentioned by this year's attendants.

In the next few days, after we have a chance to wind down, we will make the videos and other material of the conference available to others.

Well... time to go. More later.

Smalltalks 2007, here we go

Today is the day.

I woke up to the sound of thunderstorms. The first cell went by half an hour ago, and the second cell is now dropping water and light flashes around here. Hopefully they will go quieter in about an hour when I have to leave. Even then, it's just an issue of convenience more than anything else.

Registration count so far: 293.

Smalltalks 2007 --- Dinner on Monday

We would like to remind you that there is a dinner event planned for Monday, December 10, in a traditional barbeque restaurant near the conference site. The goal is to have a base price with access to a fixed menu (most likely consisting of red meat), with drinks and beverages to be paid for on a per person basis.  Although the details are not finalized at this time, we believe that the proximity of these places to Ciudad Universitaria compensates the eventual cost that may be associated.

If you are interested, please add your names here.

The reservation will be made on Saturday.  It may be the case that more room is available for people without reservations, but this is not guaranteed.

See you there!

df -k Smalltalks2007

The registration count is now 262...

... cross that out: 265.

... another update: 269.

... latest count: 272.

... yet more people: 275.

... correction: 277.

... last check before I leave: 282.

... at Houston airport: 286.

... at Buenos Aires: 291.

Hash book review is complete

I just finished going through the hash book once more. I fixed a few typos, a lot of paraphrasing, and a handful of instances of bad code. Thus I have reached the first Release Candidate PDF file, which is now 450 pages long.

Now, the mentoring course book needs to go through the same process. Currently, the draft is 324 pages long, with together with the hash book's 450 pages, brings the total pages ready to be published to 774.

More to come...

Side thought about OLPC

Every so often I hear politics oriented comments about the OLPC and things.

I think OLPC is great, because among the millions of kids that will get one in their hands, I hope we find a Gauss, a Ramanujan, or an Einstein. Giving them an OLPC is far better than what our education standards will do for them, because e.g.: the more time you spend in US public schools, the lower your IQ becomes.

If only Gauss and so many others had had a computer to write software in...