Numbers and the Next Big Thing
I’ve been waiting around for Numbers. Well, not Numbers specifically but for Apple to do a spreadsheet. Now that it’s out, I have to say that I’m disappointed. It’s not about features but about the base paradigm. I wanted Apple to revamp how spreadsheets are perceived and conceived. It may be a bit much to expect from Apple except that the new paradigm was already created some 20 years ago, and not only that, Steve Jobs had a hand in it.
Instead of explaining it all here, I suggest you read about Lotus Improv. Here’s a great article describing the history.
In short, Lotus came out with what is now known as the multidimensional spreadsheet. It was one of the first apps on the NeXT platform. It was also a revolutionary new way of doing spreadsheets.
It’s hard to really get a sense of how it works by reading about it. Quantrix has some great Flash presentations. I advise viewing those before reading on.
The main benefits of a multidimensional spreadsheet is that it actually knows about your model. When constructing a multidimensional spreadsheet, you are not constructing a visual structure so much as a semantic one. Those headers aren’t just for your benefit. The row and column headers are, in effect, axes in your multidimensional model. But you aren’t limited to two dimensions. You can define as many dimensions as you want and dynamically rearrange the axes as you see fit. The order of the axes (which axis is the column and which is the row) is just a part of the view and not the model itself. This also extends to charts. They are just graphical views of the same model and if your model changes, the charts can automatically update as well since they are based on the same semantic structure (i.e. the charts are not just one-offs).
The result of this is that the program makes many things natural and intuitive (it only sounds complicated). For instance, “pivot tables”. In the multidimensional model, it’s a natural extension of the paradigm (there’s no special marketing term for it) which makes pivot tables in traditional spreadsheets look like a hack. Natural language formulas also come, well, naturally. The generic headers and cell designations (A1, C5) are gone. You define the items in the headers so a cell is referenced as “Sales:1990” which makes tons more sense. Also, formulas are more based on the structure of the model and not on individual cells. This allows the formulas to be separated out so you can see (and edit) them all in one place (the formulas don’t go in the cells). Because it is a multidimensional model, extending any one dimension (i.e. adding rows or columns) will bring along any formulas with it. Again, this is hard to visualize if you haven’t seen it before so check out the Flash tours linked above.
So, where are the multidimensional spreadsheets now? Lotus did port Improv to Windows but Improv on both platforms ended up being abandoned. To fill the void, Lighthouse (the company I worked at) created a clone, Quantrix (which I worked on). As I’ve mentioned before, the Lighthouse apps were mothballed by Sun.
Since then, Pete Murray (one of the original authors of Quantrix) wrote it all over again, from scratch in Java, and has released it with his new company. He even got rights to the Quantrix name. As linked above, you can check it out at Quantrix (and thanks to Pete for allowing me to link to his demo presentations). Note that it is not priced for the casual user, being oriented more towards the enterprise customer but they do have educational pricing.
There’s also Flexisheet if you want something free, open source and/or native, though it does not seem to have been worked on in years.
iWork’s Numbers is fundamentally a 2D spreadsheet. It does some trickery with the headers to allow for some level of natural language formulas. It has some things here and there to simulate some of the aspects of a multidimensional spreadsheet but it’s still a traditional sheet underneath.
One subtle difference between the 2D and multidimensional models is that in the latter, the data model is expected to be dense. What this means you don’t really have unused cells; all cells are intended to have meaning in your model. It’s not a freeform grid but a packed model of data. For people used to sticking all sorts of random non-computational stuff into spreadsheets, this can be hard to adjust to. Basically, people are using spreadsheets not so much as computational tables but as a big piece of graph paper.
Numbers shifts this around a bit by making the tables a part of a larger freeform canvas. This is a big improvement from other traditional spreadsheets as I’ve always believed cells are for numbers. That clip art or paragraph of text you stuck in there is not a part of the model you are creating. It makes it such that the spreadsheet is used as it was intended and anything you attach to it, you put together with it and not in it. All in all, the separation of table and canvas is a welcome change.
Why wasn’t Numbers done as a multidimensional spreadsheet? Several factors come into play here. The main one is that multidimensional spreadsheets are quite different from traditional spreadsheets. If you’re an Excel user then you’d have to unlearn a lot of how you conceive of spreadsheets. In essence, it’s a hard sell to anyone that uses a traditional spreadsheet. The only market where it seems to stick is the financial market, which is not a market Apple is concerned with. It’s too bad, really, since I’ve always felt that the multidimensional model is actually more intuitive for the user who has never touched a spreadsheet. I felt a consumer-level multidimensional spreadsheet would have been the innovation the spreadsheet market needed.
Maybe in the end it was too much to expect of Apple. With innovation comes risk and it’s hard to bet on something that has already failed in the market once. Nonetheless, the innovation is there. The hard part is getting people to use it.
Category: OS X, Software 27 comments »
August 15th, 2007 at 2:21 pm
Given that Apple’s strengths seem to be in addressing the “fat middle” rather than the “extreme edges” of a market, I can’t imagine they’d do anything more than Numbers. I think Jobs is keenly aware of this.
In fact, Numbers appears to hit smack in the middle of the fat middle market for simple spreadsheet apps, and also raises the bar quite a bit in terms of finesse. (The Excel folks have quite a bit of catch-up to do now unless they want to look pretty pale by comparison.)
Of course, there are the Apple Pro apps, but they’re entirely in the media field, and I doubt if they’ll ever stray outside that field. They probably couldn’t imagine tackling a Pro spreadsheet market just as they couldn’t imagine tackling a Pro CAD market.
August 15th, 2007 at 2:23 pm
And, of course, that’s good for us indies–it leaves the Pro non-media fields wide open.
August 15th, 2007 at 2:27 pm
I very much like the multidimensional spreadsheet, and always hoped to see Improv catch on – largely because it broke the mould in so many areas, including cell naming. It is a shame that Quantrix is priced as it is, I can’t help feel that it could gain a much larger market if they offered a product priced for individuals. I tried the demo and really liked it (even though it is Java based) but I couldn’t justify paying out for a licence.
I really like Numbers though. It may not be as innovative as Improv was all those years ago, but it has a lot of nice touches. I think that you are right that the learning involved in using a multidimensional spreadsheet is steep. It has become more difficult to overcome since although the market of spreadsheet users is bigger, it is full of people used to the Excel model. If Improv struggled in its day, then how much harder is it now? Sadly Quantrix has settled for being a niche product, and I wonder whether we will ever see someone take the risk of making a mainstream multidimensional spreadsheet again. Numbers is expected to be something of an Excel replacement and therefore probably constrained to be reasonably similar in operation.
August 15th, 2007 at 3:30 pm
Thanks for that blog enry, it was interesting to learn about this concept of multidimensional spreadsheet, and I agree this allows for much more power.
I also just wanted to comment on what you said: “The main one is that multidimensional spreadsheets are quite different from traditional spreadsheets”. I think, ideally, to introduce this concept without having the risk of alienating users that except traditional spreadsheets, it should be simply done in a way that allows usage identical to traditional spreadsheets. Then there is no transition necessary the first time the app is used. Eventually, over time, maybe the user will want to explore more of the multidimensional stuff… if ever! It also seems to me that multidimensional spreadsheets are naturally amenable to traditional spreadsheets. You just need one extra button somewhere to display the additional few UI elements for multidimensional.
Also, multidimensional spreadsheets is a really really scary name, that is sure to turn away regular users that just want to make lists of stuff. So you would not want that to be too prominent in the feature checklist.
just my 2 cents!
charles
August 15th, 2007 at 9:24 pm
Thanks for the comments.
charles: My explanation of it sounds pretty complicated but the terms I used are not what are actually shown to the user. My POV is from someone who has worked on one as a developer trying to describe the conceptual paradigm underneath it all. I don’t think you need to be aware of the fact that you are creating a hypercube of data and creating formulas operating on vectors in that space when you are a normal user. It’s just one of those things you kinda intuit your way through when using it.
And the fundamental model is too different from regular spreadsheets. It’s not something you can transition over easily. It looks superficially similar but it’s not a matter of extra UI. Even with just two dimensions, there are big differences in the behavior that jamming both paradigms into the same spreadsheet product would probably not work. Plus again, the more time spent in the traditional model, the harder it will be to transition.
Chris: The odd thing here is that you have a product that caters to either the hardcore or the super novice. Every spreadsheet seems to cater towards the middle. It seems that possibly going after the market Excel doesn’t would be the way to go. In any case, I seriously doubt Apple will go after the “pro” market in this case since it’s enterprise customers which Apple has shown little interest in.
Karen: Drop the Quantrix guys a line and let them know. Maybe if there’s enough interest, they could do a consumer-level “lite” version. Also, Flexisheet is open-source. While it’s not easy getting a bunch of people to work on it it’s a definite option if you or someone you know is interested.
August 16th, 2007 at 2:36 pm
The concept is called a database; you might have heard of it. There might even be one for your limited toy-computers.
August 16th, 2007 at 2:42 pm
Oh yes, I’ve heard of databases. And if you actually understand what is being discussed, this is not a database. And yes, there are quite a few databases available for my PC but that’s besides the point.
August 16th, 2007 at 2:45 pm
I really like the Multidimensional spreadsheet idea. However, how is this different from something like MS Access? (Except that it might actually be usable). It seems to me that a multidimensional spreadsheet is more a database than a spreadsheet (i am using the term as understood by most in the post Excel world).
August 16th, 2007 at 2:56 pm
vrad:
Multi-d spreadsheets are still spreadsheets. Just as Excel is a computational front-end, so are multi-d spreadsheets. If you check the Quantrix site I linked above, you’ll see that their product can be backed by a database but it’s not necessary.
Basically, it’s still a spreadsheet. It performs the same function as Excel, just in a different way.
August 16th, 2007 at 3:08 pm
I was also a little surprised that Numbers wasn’t a reborn Improv, but this is the first release.
I’d not be surprised in the slightest if Numbers ’09 IS multidimensional, and uses more than a little flourish of CoreAnimation in order to add a little flair to viewing the data.
Personally, though, the lack of external data source support is a bigger shame to me: the canvas UI would be perfect for manipulating a dynamically-refreshed data source. Imagine a Numbers table that’s a view of some data somewhere, updated on the fly, with the charts (and other tables) around it updating as the source data does.
August 16th, 2007 at 3:17 pm
Love the Improv method aka multidimensional spreadsheets. It really does make sense and is very useful.
Man, what a trip down memory lane… I remember when Lotus introduced this on NeXT. Sheesh… that was a long time ago!
I’d like a mix between this and Numbers.
August 16th, 2007 at 3:45 pm
Excellent post. I use a multi-dimensional database at work called Essbase. It’s a Hyperion (now Oracle) product.
The cool thing about Essbase is that has a plug-in for Excel that allows you to interact with the data and produce very slick reports. Definitely a spreadsheet on steriods!!!
We use Essbase for various financial planning and reporting functions. It easily handles and manages large amounts of data.
But, it’s definitely not a consumer application. The base Essbase product costs several thousands of dollars. Not exactly in my budget.
August 16th, 2007 at 3:50 pm
I was also hoping that numbers would build on the Improv tradition. Still, I find it’s interesting that Numbers ’08 doesn’t have pivot tables. Now, I wonder if within the new Numbers spreadsheet paradigm of tables on canvases if it might be possible to add a multi-dimensional table to a Canvas. This would provide a powerful tool for data analysis, but without making Numbers anymore complex to people more familiar with 2D concepts. It might work somewhat like a pivot table and thus allow for a degree of import export compatibility with Excel, while implemented in a more Improv UI/data model style.
At any rate if I were an Apple product manager or engineer this it the direction of innovation marketability that I would pursue. The technical changes might be too difficult to pull off, but from a user perspective it would appear to be a more natural innovation. Just as the tables on a canvas model is both understandable and puts excel’s formatting abilities to shame, the option of adding an “improv table” (or whatever name apple might come up with) on a Numbers canvas would put excel’s pivot tables to shame. Yet, as the feature would be compared to pivot tables, it could be marketed as “pivot tables for the rest of us.” And it would have to presented with use cases for average users, making advanced financial easy. Perhaps, some templates could be created for the SOHO market, for sales analysis and stuff like that. Likewise, with so many people investing on there own you could create a template that would help you analyze company financials. In fact, if the template could pull data off the web this short of feature could be really useful for the average person.
Anyways, I hope that someone from Apple is reading this article and its comments!
August 16th, 2007 at 4:17 pm
The whole model reminds me, just a bit, about Dabble DB. Do they (the Improv lineage and Dabble) share anything meaningful or is my mind trailing of in the wrong direction?
Great entry!
August 16th, 2007 at 4:49 pm
Till Apple doesn’t resolve the quality of Numbers at least up to Clariworks Spreadsheet – Multidimensional tools are as much away like the Mount Everest for ordinary Hikers.
NeXT had great Spreadsheets even “simple” powerful ones like Mesa or WingZ not to mention
Parasheet.
August 16th, 2007 at 4:58 pm
I thought I was the only one that rembered and missed Improv, and I didn’t even new that there were similar spreadsheets in development. I have been expecting a return to the concept, that seemed to never happen.
A few years ago I managed to find a copy of Improv in french in the eDonkey network and kept it as a memento, but never managed to use it. I’m not sure it was declared abandonware…
August 16th, 2007 at 6:43 pm
Yep. DabbleDB is the way to go for us Improv-heads. It’s written in Seaside/Smalltalk, and does the multi-dimensional paradigm as a web app. Would be nice to have it on the desktop, but I don’t see it happening. And Apple are *not* going to go there.
August 16th, 2007 at 6:54 pm
That Quantrix site is hilariously square and corporate (about what you’d expect from people who thought Quantrix was a good name for something other than an heart-disease pill).
Do even business people like wading through that stuff? Are they sitting there going, “This software will let me leverage my synergies? AWESOME!!!”?
August 17th, 2007 at 9:30 am
Seems if Apple wanted to add this to Numbers in the future, they could add it as another object type in the overall layout space of the canvas. I agree that they probably wouldn’t see the need.
Cool approach though, I’ve not seen it explained before.
August 17th, 2007 at 10:16 am
I am currently using Aabel from Gigwiz (www.gigawiz.com) which is working exactly as you described and is a really optimized and polished application. The price is also hard to match. I am surprised that it is not more popular. I think that the target audience are mainly scientists, but the plots look really impressive, it builds on a real-time multidimensional structure and even features a database metaphor. It is Mac-only.
August 17th, 2007 at 11:15 am
Thanks for the heads up on Dabble DB. Hadn’t heard of it before but judging from the demo video, I don’t see much concerning the calculation engine. It seems more like a visualization tool for relational data.
Aabel seems particularly oriented towards graphing series of data points but it’s hard for me to tell from their site how close it is to multi-d spreadsheets.
As for Apple adding it in the future, I don’t think that would work. The paradigms are different enough yet they look superficially the same enough that having both in the same product would spell mass confusion. As scary as the whole multi-d thing sounds, when actually learning it as a total novice, I believe it to be more intuitive than the traditional model with the important caveat that the user has never been exposed to a spreadsheet before. I’m throwing out all this wordy analysis and the paradigm has been worked out rigorously to where it’s almost like it’s own math but users don’t have to know any of that to use it. The problem is that once they pick up the Excel/traditional way, moving to multi-d is much, much harder.
Basically, we have the odd situation of a product that works for both complete novices and hardcore users, but not the people in the middle. The current products shoot for the upper part since history has shown that there is uptake there. Question remains whether the product can be marketed towards the novice users but for now it seems that Apple is shooting for that creamy middle.
August 17th, 2007 at 1:53 pm
Could someone elaborate on the difference between a multidimensional spreadsheet and a database?
August 17th, 2007 at 2:55 pm
Apple’s market is MS Office switchers and AppleWorks refugees. At this time, numbers has to do exactly what 95% of Excel are already doing, only easier and better-looking.
If the tables-in-panes interface becomes familiar to users, then perhaps an optional multidimensional back-end can be retrofitted in Numbers 2.0.
October 11th, 2007 at 5:38 am
Mac OS X also has a “simple” powerful one called Tables, http://www.x-tables.eu/more/overview.html .
March 5th, 2008 at 6:59 pm
Late to the comments, I am, but I was so impressed with the Apple Numbers app that I was re-inoculated with the multi-dim bug from the old days of using Lotus Improv. Of course, I ‘fess up to actually liking Lotus Symphony in 1988 and being an actual, bona fide user of Lotus Agenda in 1990.
Well, I searched the net; found a mint condition, never-opened version of Improv and bought it this afternoon. Now, if I can just find a PC with Win 98 running.
That aside, we should probably lower our expectations for Numbers heading toward multi-dimensional modeling. Guys: the whole iWorks package sells for less than some of my son’s video games.
June 18th, 2008 at 11:02 pm
After playing with Quantrix today and reading the comments here, I tried out Citrin (Aabel’s demo isn’t downloadable). There’s a definite difference in feel between the two; Quantrix emphasizes direct manipulation of the metadata (axes/dimensions), whereas Citrin allows direct manipulation of the visible data, yet when you move into metadata, it shifts to buttons and switches everywher. For example, you can drag axes or items in a chart legend to reorder them in Quantrix, versus picking from lists in an inspector in Citrin. It looks like Aabel is much the same; take a look at this video for example:
http://www.gigawiz.com/movie3r.html
While it’s admirably usable for a Java app, I wish Quantrix would adopt a few useful Mac OS X behaviors—the behavior of its pseudo-inspector (“Format Toolbox”) is confusing, and the inability to copy/paste styles—or define templates at a granularity smaller than a document (“model”)—are annoying when you want to analyze different instances of data in the same way.
July 20th, 2015 at 2:26 pm
Years have passed since the latest comment, but I suspect everyone here would be extremely interested in ClearFactr. It uses plain language formulas that are easily and intuitively expressed at the cell level, but then does automatic reporting of the similarities and exceptions at the model level. It then leverages those benefits via 1-click analytics that replace hours of tedious, error-prone, spreadsheet-risk inducing programming. All in your web browser via SaaS. Everything’s versioned too, with scenario comparisons built-in, so it’s kind of like a GitHub for spreadsheets.