Analysis


23
Mar 10

The Need for Custom

Progress can be measured by how little we do the same thing over and over again. Once you have a process, you can wrap up each piece in a little section, hand that to someone and then press the button that says “go faster”. Things generally work this way. The first car took awhile to put together, but once some guy figured out what pieces you need to build, he realized you could automate and streamline most of the work. Doing it the same way every single time made the price way lower, the ease of maintenance much higher, and resulted in many more sales of the Model-T. As for customizing things, Henry Fords famous line was that it came in any colour, so long as it was black

On the flip side of things we have a culture of consumerism that is very motivated to satisfy their need for self expression. This desire to  be ‘different’ and express ones individuality is often the motivation behind many purchasing decisions. It influences your decision about what car to drive, or what computer to buy, what software to purchase, or what sweater to wear. Every so often a counter-culture will erupt whose whole purpose is to be hyper-individualistic, hyper-free, or to challenge the status-quo which, ironically enough, is actually a method of self expression in itself.  It’s important to note that Companies are like people in this respect as well. Corporate culture and habit dictates a lot of their purchasing and ‘innovation’ decisions.

What if the need to make things just ‘a little different’ is over. What if we have too many choices? What if we just bundle up all the bits and pieces into something reasonable and say “You can have it in two colours: black and black”

We have two very different systems on our hands. One hand suggests that in order to make a product economically viable you need to have a process which makes millions of them. On the other we have a value system that suggests we customize and change things – in order to satisfy our desire of self expression and individuality. Where one decides to offer the ‘customizable’ part is what will make or break your business (unless of course, you’re in the business of customizing).

The placement and degree of customization have a lot of big implications.

  1. The more different you make something, the more combinations of shit going wrong you have to predict and support.
  2. More permutations = decrease in instantaneous knowledge transfer. For example, if you could make your iPod work 1000 different ways, when you hand it to your friend they’ll just stare at you and blink – nobody cares if it’s just covered in sequins.
  3. Hyper customization results in a situation where you also can’t produce enough of something reliably or cheaply, which results in all sorts of messes. This is particularly interesting in software. What gets included as a feature? What ends up just being a plugin?
  4. Usability and interaction / retention can be severely impacted. Where do you draw the line between custom color schemes and layouts, and being able to personalize & express yourself through your purchases instead.

(There are a lot of things these points can be tied into as well. For example, it seems as though people are much more willing to accept less customizable control over something if they’re subscribing to a service or purchasing an object that is indistinguishable from magic)

While many industries are designed with the understanding that we need things “just a little different” it also wastes a lot of resources and in some extreme cases delays progress. While I’m a firm believer that monopolies and globally homogenous environments are a bad thing, it’s interesting that the most successfull companies I know of today are the ones that have convinced people that they dont want things different; they want things the same.

It’s the Model-T all over again. Only now you can get your iPod in a few more colours.


18
Feb 10

Carbon Computing

Being able to outsource all of your computing needs to an external provider is absolutely fantastic for developers and some businesses. While not running your own infrastructure is much cheaper, it also makes the amount of energy and associated ‘costs’ of computing very opaque.

There are quite a few issues with the transparency of costs in the cloud computing space. This includes no transparency into the cost of electricity, and where that electricity is coming from. To date, there is no public database of electricity markets, datacenters, and hosting providers which lists how much carbon per kWh of electricity is being output. In ICT this is a massive issue, because of the sheer amount of energy our industry consumes.

“Information and Communication Technology (ICT) is both a problem and a potential solution in the war against climate change. Currently, computers are responsible for more greenhouse gas emissions than airlines. Greenhouse gas is growing exponentially and we expect that ICT will produce double the emissions of the airline industry within five years with no end in site. ICT can provide a solution to climate change by reducing carbon emmission in the world through telecommuting and other means.”

Here in British Columbia we get most of our electricity from renewable hydro-electric power. Hydro-Electric is one of the lowest forms of reliable low carbon output electricity generation available. It goes without saying that providing computing services using energy generated this way would mean less CO2 / kWh but also less CO2 per compute cycle.

We need several things to make this happen

1) Start measuring how much power ICT is using on a per server / component basis
2) Develop resources that track carbon output per kWh in different states & provinces and provide that information as a service
3) Determine where your computing resources are located and track on a per machine level the amount of carbon being output
4) Calculate how much carbon you’re using.

Those are pretty audacious goals, but I think we really need to start keeping track of carbon output for power. By tying that into the different services we use on a regular basis, we can make carbon part of the social and actual cost of using services. Hopefully that will help buy us enough time and money to develop the carbon neutral power solutions we desperately need.


22
Apr 09

URL Shortener Statistics on Twitter

URL Shortening services have gotten a lot of attention because of Twitter. The reason is that most significant links on the internet are longer than 140 characters and that doesn’t leave you any room to actually say anything when you tweet. Twitter will shorten some URL’s for you using TinyURL, but what about all of the other services? What’s the state of the URL shortening service world on Twitter?

Of course, the best people to answer this question is Twitter themselves, but since they haven’t published anything, and I doubt they’re interested in spending time answering my almost academic question I decided to write a quick app to get the stats myself.

The Twitter API is gorgeous. I mean. It’s simple, easy to use, and it’s fast. I grabbed a list of all the popular URL shortening services from a few locations, cross-referenced them and came up with 36 “popular” services. The actual number of URL shorteners is pretty incredible. (Some of them even have long names like “shortna.me” WTF are you thinking?)

The process is simple – Search for the urls of the shortening services using the search API, then count how many hits you get. Repeat every 30 seconds or so. (There are other nuances – look at the code).  This sampling was done over a 3 hour period on Wednesday April 22nd between the hours of 8AM and 11AM Pacific.

Results

Twitter URL Shortening Statistics - April 22 2009

Twitter URL Shortening Statistics - April 22 2009

Issues

You can only return 100 search results at a time, which means if you see 100 results for your term, there are probably way more of them. The only way to get around this would be to search more frequently for that specific term. I tried doing that and was quickly throttled by Twitter. Suffice to say, TinyURL is still the dominant force here, but probably because of their Twitter integration. This really throws off the numbers. There’s a fine line between searching too often, and getting accurate results. Tests in the middle of the night showed that TinyURL had about 30% of the URL Shortening Service market on Twitter. I’m 99% positive that TinyURL numbers are way higher.

Things to do

Optimize the searching algorithm so it displays more accurate results.

Record the time, and see statistics over the course of hours, days, weeks. I can tell you off the top of my head that TinyURL usage is high all times of the day, where as bit.ly usage is pretty much a “waking hours” service, as most others.

Better Reporting

GitHub

The scripts are on GitHub. Go bananas. http://github.com/Trevoro/urihz/tree/master