What We’re Not Admitting about Asterisk
It’s time to clear up several restraining misconceptions in the telephony space. These are points I outlined during my presentation at AsteriskTag in Berlin recently, though they bear repeating for everyone’s sake.
This post is not the end of the discussion. Please feel free to leave comments on this post and subscribe to my RSS feed. This post gets a lot off my chest but I’ve a growing list of things to address very soon.
If there’s one thing I want you to understand, it’s this:
Asterisk Has Not Crossed the Chasm Yet.
I’m referring of course to the chasm in the technology adoption curve, best illustrated in the graph above (after which Geoffrey Moore’s book Crossing the Chasm was named).
To those unfamiliar with the concept, it defines five demographics of technological adopters. The first two, innovators and pragmatists (visionaries), give a new technology its first breaks; the bleeding edge is either a passion or a competitive advantage to them. Beyond these demographics exists a chasm which every technology must suffer and overcome to enter the lucrative early majority market.
One fascinating characteristic of technology adoption (which Geoffrey Moore introduces early in his book) is the false sense of security it can create when the rush of enthusiasm comes from the early adopters (left of the chasm). To the company, it’s easy to assume this adoption will continue linearly into eventual ubiquity. This misconception has crushed many a dream when the unsuspecting company ramps up just as their adoption profile plateaus at the chasm. I feel Digium has been caught up in this misconception.
I’m convinced because, in my 2.5 years of trying to make Asterisk better, I keep seeing the same kind of person approach me either online or offline at a conference. Let’s call him Otto. He’s the young, inspired hacker fascinated with the thought of controlling a phone. I was Otto once. He finds documentation, overcomes the installation, gets a “Hello World” working, then stops. Beyond that point Asterisk becomes prohibitively unintuitive and that impression sticks and becomes a reputation.
This quintessential newbie we’re calling Otto can then do two things: leave the community altogether or stick with it and complete a single project or even series of Asterisk projects. My experiences in the open-source telephony space have revealed two observations of mine you should consider:
The average amount of time a developer plays with Asterisk is less than one week.
The developer retention rate of Asterisk is less than 1%.
These observations encompass everyone who has ever considered toying with Asterisk and taken the first step. A first step may be an “apt-get install asterisk”, buying the O’Reilly Asterisk book or even cracking it open online. I’d love for you to prove me wrong on these with real market research. I actually think the median time spent using Asterisk is just one weekend and the retention rate is less than 0.1% when you consider international hackers (like those to whom I spoke in Germany recently). I passionately care about this issue because, for the time being, open-source telephony and Asterisk are virtually synonymous. I want to see the entire ecosystem ramp up their expectations. And I want to help us win this fight for the advancement of technology around the world. That’s really what’s at stake.
The error is not a marketing error. Digium is a multimillion dollar company with a clear image: they make hardware and appliances for the SMB PBX market. This brand extends to the open-source Asterisk software which inherits the reputation and the hardware from this umbrella company. People understand what it is. Otto was already sold when he dove in.
The error here — the reason why Asterisk has not crossed the chasm — is technological.
Digium and its surrounding ecosystem has not engineered a Tipping Point. Specifically, they have not engineered Asterisk to be “sticky” enough to spread like wildfire, a task which should be the core to any open-source umbrella company. After all, open-source’s strongest advantage is its propensity to cause a wildfire effect. Many companies open their source because it’s free help that has an inextricable marketing effect. (I’m loosely citing Malcolm Gladwell’s book The Tipping Point here, another must-read for technologists.)
So what the hell is wrong here? Below I enumerate four more misperceptions that I think are immensely damaging to the Asterisk ecosystem and are preventing Asterisk from becoming “sticky” enough to cross the chasm.
Truth: If You Build an Asterisk System, You’re an Asterisk Developer.
An improper (and more common) way to phrase this sentence would be “If you configure Asterisk, you are an Asterisk user.” Though only a subtle terminology difference, thinking of Asterisk “users” as “developers” opens the door for us to treat them as such.
Geeky internet citizens can likely remember seeing at some point the notorious video of Steve Ballmer running around a Microsoft stage screaming “DEVELOPERS! DEVELOPERS! DEVELOPERS!” Microsoft is starting to understand what has empowered the open-source world to become an unstoppable deus ex machina: it’s about the developers, man! Open-source is a philosophy which places developers at the forefront with the fewest restrictions. And it’s extremely successful! Last year Steve Ballmer said with a straight face “I would love to see all open source innovation happen on top of Windows.” He’s an enemy and, because he understands this principle, he’s an especially dangerous enemy. Microsoft has entered the voice market and is now competing with Digium. And Microsoft’s even doing many things right technologically! We need to react.
We can react by treating users of Asterisk as developers, not system administrators. This is a technological truth that has been glaringly obvious but shamefully ignored. By “developer” I do still mean someone who touches only the “configuration” files which Asterisk provides, not someone who modifies the C code with which Asterisk itself is written. I call these people developers for the following reason.
Truth: When You Write extensions.conf, You Are Writing Software.
To those unfamiliar with the technicalities of Asterisk, extensions.conf is the main “configuration” file in which the dial plan is written — the logic that controls the procedures that occur during calls that go through your Asterisk system.
My stating that extensions.conf is software always gets an amusing reaction from the average Asterisk hacker. It usually goes something like this:
“When you write extensions.conf, you’re writing software.”
“Well, extensions.conf started out as a very simple solution to solve Asterisk’s initially simple problems. It was not designed to be a full-feature programming language.”
“But it’s now a language with functions, control statements, loops, labels, conditions, variables, even macros and subroutines!”
“Yeah, but it’s not a programming language. It wasn’t mean to be.”
I’ve heard this from several people over and over. The Asterisk hacker with whom I’m conversing generally concedes the point after various facial color changes. Let’s at long last admit this, guys: the extensions.conf grammar (and the derived syntactical alternative AEL) are very poorly designed dynamically-typed scripting languages. Someone who modifies this file is called a programmer. The extensions.conf file is what we call software. A .conf file extension should only be used for configuration files. With these (irrefutable) assertions in place, we can start to frame the picture of what the next approach should be.
Our oversight of this subject has had an interesting effect. I’d say the biggest demographic of Asterisk users is the sysadmin type who feels no anxiety toward a “configuration file” but may feel a little anxiety toward modifying a “.c”, “.java”, “.rb” or “.py” file. This distinction between a .conf and “real code” is mostly superficial. I say “mostly” because, when the Asterisk developer must modify a C file which performs the same function as extensions.conf, the distinction is greater because she knows that there are a lot of ways to easily screw up the entire PBX, having disastrous consequences. This is exactly why Asterisk’s monolithic voicemail system has gained a bad reputation: it’s written in C and difficult or dangerous to modify.
When we acknowledge that extensions.conf is a scripting language, one abstracted deeply from the dangers of the machine language paradigm in an attempt to improve productivity, the discussions to “improve the Asterisk developer experience” inevitably conclude that a real scripting language must be used. For today’s and tomorrow’s complex uses of Asterisk, the scripting language in place now completely breaks down on several fronts.
This is not a blog post about Adhearsion, but this is where I have extensive experience on this subject through my years implementing it. When we take the first step and begin developing scripting language support for a telephony platform, I’ve found that what’s also really needed is a framework that further abstracts the legacy Asterisk paradigm into what should be a DRY (Don’t Repeat Yourself!) user experience. The framework can provide a plugin system so people can trade small units of functionality at little risk. The framework can allow better bundling of those applications. The framework can make deployment assumptions. The framework can evolve and provide new framework-level hooks into other disparate technologies that were previous unthought of.
I’m extremely passionate about this subject and can continue extensively on it. Let’s save that for later and continue. Grab my feed if you’re interested.
There’s a Difference Between an “Asterisk Application” and an “Asterisk System”
On this point it seems I diverge completely from the Asterisk status quo. During Mark Spencer’s great keynote at AsteriskTag, he gave a fun list of Asterisk applications as a way of inspiring the community to innovate. The list included about nine Asterisk “applications” such as the Botanicalls project to receive phone calls from your anthropomorphized plants when they’re thirsty, a tongue-in-cheek queue game “application” that makes people waiting in a queue play a game instead of hear music (winners move up in the queue), an application for pilots to get aviation reports from any cell or landline phone. These are all great examples of using Asterisk, but they’re not “applications.”
These are examples of Asterisk “systems.” A non-trivial use of Asterisk (such as these) usually requires many custom shell scripts that adhere your program to other technologies. Frequently they’re tightly coupled to machine-specific assumptions such as the Linux distro choice, use of CURL, directory paths, sip.conf trunk names, extensions.conf context names, hardcoded database information, IP addresses and so forth. This tight coupling prevents easy distribution, therefore no one trades implementations and everyone reinvents the wheel.
These “systems” of interlocked clockwork are required because of the deficiencies in the extensions.conf scripting language. If dial plans were written in a high-level language, there would be no need to shell out to accomplish some small unit of functionality. Everything would be a part of your dial plan software. For this reason, the solution to this tight coupling is for a scripting framework to standardize what’s most commonly done in an abstracted, high-level way. Features that were not thought of are implemented as plugins. Plugins are shared and configured to achieve the desired effect.
I think of an Asterisk application like a desktop application. If I want a call center with a sophisticated, fine-tuned web interface that shows agents prompts with AJAX from a MySQL database, I should be able to go online and click a “Download Now” button somewhere. If I run a skydiving dropzone and want an application to manage the jump schedules with telephone reminders, I should be able to go online and download something pretty close to that. This expectation of “yeah, someone’s probably thought of that — let me go check online” is one that we’ve all grown familiar with in the open-source economy. For desktop software, this expectation of abundance is fundamentally powered by the ease with which desktop applications are distributed. For telephony software, this is completely not the case.
This is also one of my motivations behind Adhearsion’s design. It should be a default expectation of an Adhearsion app that the author wants to eventually distribute it online and help build a greater telephony application community. If they don’t want to, that’s fine, but I shouldn’t stop them.
And if an open-source economy emerges around telephony applications, we’ll begin attracting people who would have never otherwise thought that they could actually use telephony to their advantage. By increasing our reach and relevance, we’ll be engineering a Tipping Point.
Telephony is Too Complicated for Every Hacker to Have a PBX in Their Closet
This is something that Asterisk has little control over. Someone building an experimental personal application inevitably must use one of these two technologies: SIP trunks or a POTS (Plain Old Telephone Service) line with an analog telephony PCI card. There are fundamental problems with both of these approaches unfortunately. I’ll start with SIP trunks.
SIP trunking is a way for any computer to terminate a call onto the Publicly Switched Telephone Network by using a premium service. The VoIP software you’re using (e.g. Asterisk) places a pure-VoIP call to this service using your account credentials. With metadata that comes along with the VoIP call, the service operates the other leg of the call and terminates your packets in a way that can reach any phone in the world.
The problem with SIP trunking is simple economics. The people making the real profits in PSTN termination realize it’s not worth the overhead to deal with small fries who push only 1,000 minutes a month through their service. Instead, they’re looking for three orders of magnitude more usage. Small fries like you and me get service through people who resell the real providers’ services and operate at scale. Most companies you’ll find online for SIP trunking are actually three to six companies downstream from one of the multi-billion dollar companies that are really doing the terminations.
The effect is poor support, poor quality, and poor features. It’s a race to the bottom and the solo telephony hackers lose most.
One thing that’ll definitely work reliably is an analog card (like those that Digium sells) that handles the analog signaling into the landline phone jack on your wall. Unfortunately, these require a considerable financial investment and, by nature, don’t spread like wildfire. One can’t take a wild ride through the world of telephony development for a weekend as just an experiment when one must to drop hundreds of dollars and spend hours configuring Linux kernel modules to get started.
I vent this here because it’s stifling the Tipping Point of open-source VoIP and there is a solution. I’ll mention that in my conclusion.
Most people want something other than a PBX
I’m going to cite another technology book to make my point here: The Long Tail by Chris Anderson.
The Long Tail is a graphical concept which visualizes diverse products in the same market. It’s a fascinating subject and is perfectly applicable to the telephony space. There are certain applications that everyone wants: they form the Big Head. Fixating on this space is the orthodox business model. Then are are thousands of niches in The Long Tail that have legitimate business needs but the demand for a product that fits their needs is much smaller. In the past, production costs made producing for these niches economically infeasible. Now, open-source, VoIP and the World Wide Web has made it possible to not only develop these niche applications on a dime but actually reach the customers as well!
This is where the importance of the application comes back around. The PBX, albeit important, is just one application and the aggregate demand of non-PBX voice apps exceeds the demand for PBXs. Voice is something that’s now universally accessible (thanks to cell phones) and universally understood (everyone understands how to make a phone call). Given the needs of the entire world in this space, it’s a no-brainer that the SMB market is a minority. If you can apply voice to a problem creatively, you can spin straw into gold. Consultants take note.
The Manifest Telephony Destiny
It’s absolutely inevitable that voice applications saturate the Long Tail over time. Thanks to the internet and the fruit it has borne, the niches will become much less ignored. Let me call this the Manifest Telephony Destiny. Just as the pilgrims settled along the east coast of America and dreamed about spreading the country’s borders across all the way to the Pacific Ocean, we’re now the pilgrims who’ve settled on the Big Head of the Long Tail graph peering toward and dreaming about what lies to the right. It’s our Manifest Telephony Destiny to reach across far, far to the right of the graph. If you’re a businessman or businesswoman in voice today, understand this principle and believe it. Talk about it at your next board meeting. It’s coming, there’s money in it, and it’s about to pass you up. Open-source telephony is the enabler and the experts will reap the rewards.
The Bottom Line
Hopefully this post shoos away the aforementioned elephants in the room. We must change these perceptions because they are damaging to the open-source space and, by association, the entire world. I really believe that.
I’ve been thinking very critically about this industry for over a year now with my entrepreneur hat on. Over the past eighteen months I’ve been to over fifteen states and six foreign countries almost entirely on Adhearsion-related business. I’ve talked to many industry visionaries, newbies, respected technologists, entrepreneurs, and the technologically illiterate. This post crystalizes some of the lessons I’ve learned on this amazing journey, but an even more amazing journey awaits in the new company I’ve formed to ameliorate in a fundamental way my many frustrations with this industry.
Here’s what I’ll let out: its name is Adhearsion Inc. and it’s not consulting. We’re currently seeking private funding. If you are or know of investors, send me an email at jay-at-adhearsion-dot-com. We think we know how to turn the industry around.
Stay tuned for more from my mind. 
Hi,
I’ve been working last year on a VoIP project for my diploma project for school using Asterisk and the Cisco Callmanager.
Besides other things, we’ve scripted a small Rails contact application using Juggernaut and RAMI. So you could click on your contact, and the PBX originates a call to the person you’ve clicked on. If you had our web-application open you’d have been notified if a new call comes in and some information about the caller would have been displayed.
Such CTI-apps are really interesting, since you can integrate a whole stuff of other information with it. You might e.g. combine them with your groupware-software and display your appointments of the person who is calling you, or integrate it with word/excel so that you can easily call the person who has written the document if something is ambiguous for you, …
Unfortunately the whole stuff was actually very painful to implement on Asterisk, especially tracking if a redirect happened and some similar stuff. The whole AMI-notification messages are in my opinion hard to track (at least compared to the Cisco-JTAPI stuff Java-Event-Listeners), especially if you e.g. want to know the callerid of the other person calling you, before you pick up the phone, since the whole notification-events are somewhat channel-centric :/.
Thus it would be of course really nice if there was some kind of higher level framework which handles all the dirty stuff for you. And of course it would be great sharing applications on top of a framework :).
On the other side Asterisk is really great when it comes to generating your configuration automatically from e.g. LDAP (e.g. generation for your voicemail-config, …), since they are only text-files and you don’t have to mess around with some kind of SOAP-API like the CCM does. I personally have always loved the idea of having an programmable dialplan in which you can even call scripts from outside via AGI, which can do all sorts of things. Using Ruby and Adhearsion for it instead of this strange scripting language makes the dialplan very readable.
Besides the configuration and API issues, it would be nice if there was some support for SRTP and SSIP (actually, I think this was recently added?), like other “big” PBXes have.
Being a hobbyist, I also know the problem with the PBX at home very well. Besides the POTS and SIP access, you might also want to buy a SIP hardphone (for e.g. developing XML-services, which are btw. very cool too
), which are also more expensive than regular phones :(. As a result of this (and being a student -> always short on money) I stopped scripting VoIP-applications, as the project ended.
Kind regards
Nax
10 Aug 08 at 10:51 am
Great essay. Configuration/programmability of Asterisk is probably the final straw/barrier for the 99% of developers spending their 1st and last weekend hacking VOIP. The 1st barrier is the almost impenetrable array of terminology and acronyms for telephony newbies.
A replacement/overlaying framework for Asterisk should consider lowing the barrier to noobs who do not know the terminology. Instead, embrace their ignorance and either permanently hide the terminology and/or teach it to the new ppl as they learn the new framework.
Similarly, with the aim of nailing the tipping point the new framework can make some assumptions:
* new user does not have a VOIP handset nor softphone yet (include the latter in the installer)
* new user does not have a VOIP->PSTN plan (include a simple, international signup site)
* new user does not have a dedicated Linux box lying around (though it is fun to point to a box on your floor and say ‘that’s where my phone calls go’)
* new user wants outbound phone calls to work out of the box (”oh, this thing can make phone calls!”)
* new user wants to receive phone calls (even without a PSTN phone number, perhaps a Skype-like ring-me test service could be provided to all SIP users to reenforce in their minds that their setup is working)
* new user wants to be able to rejoice and show off their simple setup; bloggers need something to blog about; twitters need to tweet.
Now you have positive viral messages bringing in new developers.
Plus you need all the stuff you mentioned above, like plugins/components/systems.
I’d love to leave the 99% of one-time weekend Asterisk hackers (I think I lasted a couple weeks back in early 2005). But as you said in your essay, the cost of becoming a part-time telephony hacker is too high to ever become a full-time telephony hacker -> developer -> professional industry consultant.
Dr Nic
10 Aug 08 at 5:01 pm
I have one word to say for starters: BRAVA!
I must admit, this article is one of the only articles I’ve read that hits the nail. While the Asterisk “developer” eco system enjoys a prolific number of “developers” as Jay describes them, I believe that the eco system needs to be better granulated.
I live in Israel, which had turned into a major Asterisk PBX adopter. Israel is about 800Km from one side to the other, and we have over 15 different Asterisk PBX integrators - each one with their own version of FreePBX/TrixBox/PIAF. Apart from these 15 vendors, we over a multitude of would-be Asterisk developers, I usually refer to these as “Extension Jockeys”. I admit the term is slightly insulting, but in my perspective, there is a high difference between one who writes dial-plans and one that develops multi-part Asterisk systems, that interconnect with multiple components.
The main problem here is when you go about and define the “Extension Jockeys” as “Developers”, your potential market suffers. It suffers because a “Jockey” is bound to do the work of a “Develoer”, when it is clear he has no idea of what to do. For example, I surely don’t expect a Jockey to inter-operate XML-RPC, Daemons and web-services into a single Asterisk application, simply because he usually lacks the knowledge.
I agree with your description of the “Developer”, but I believe that a better granulation is required.
Nir Simionovich
11 Aug 08 at 6:23 am
To some extent, the point of Adhearsion is to make Nir’s distinction moot. The long tail is full of opportunities that do not care about what the network is doing on the back end. Adhearsion opens up things for the innovator and entrepreneur who can hook into an asterisk blackbox and start populating the long tail. There will be plenty of work for high level developers in the big head space, but adhearsion is about getting the thousand flowers blooming.
Jockey
11 Aug 08 at 1:46 pm
I agree with what Jay is saying, with 2 reservations:
1) don’t ignore the administrators - they are a separate group
from developers and need to be treated as such - think about databases and DBAs.
2) Web 2.0 is the place where developer energy is. Asterisk needs to capitalize on this, in 2 ways:
i) we need to adopt a web tools and techniques so web developers will feel at home immediately in asterisk. At the moment Freeswitch is much further ahead in this respect. You can give a web developer a freeswitch project and you will get something back. Try that with asterisk and you will wait a month and most likely get nothing.
ii) We need to make it easy to integrate telephony apps into web applications. In almost all of the niches the telephony is just an extra medium (possibly not the most important), the primary interface is the web.
- there - thats my rant!
steely-glint
12 Aug 08 at 12:35 am
Brilliant post.
When I was working with the Callweaver folks I really lobbied for some kind of embedded object oriented language we could use to simplify the “dial plan”. I wanted us to have a library of drop in “dial plans” you could use for your home, small business, or call center. Alas, re-engineering Asterisk is hard enough without adding features and me with no C development skills to speak of.
The real market for Asterisk right now is where people are creating products of Asterisk as a PBX. They haven’t moved much beyond this. In the future Asterisk or something like it will be the “web Server” of voice applications but the tools are still being refined, just like in the early days of the web with perl-based cgi. Voice recognition and internationalization are challenges that the tools don’t live up to yet.
I only hope the sad day does not arrive when the .Net extensions and tools for voice are mature are we are all standing around asking what the hell happened? We owned this space!
Nate Smith
13 Aug 08 at 11:45 am
Fantastic post. I count myself lucky to be in the 1% that got past the first weekend, albeit just for a 6 month contract.
Nate - I don’t think M$ will ever take this space, they will always charge for product and there will still be pstn connectivity costs.
Of course there are alternatives to Asterisk, but only SIP servers. The pstn network will be around for some time yet.
I wonder if web 2.0, 3.0, 4.0 are the future for Asterisk or if telephony is a different kettle of fish?
best thing about the post is that it has got me interested in Asterisk and Ruby again…
Jay Greasley
13 Aug 08 at 1:31 pm
Recently Jay Phillips of Adhearsion fame sent me
a blog he’s written about Asterisk and asked me
to respond to it, which of course I’m happy to
do. First, I have to say that I met Jay at
Astricon and was immediately impressed with his
energy and excitement around Adhearsion. To say
Jay is an out of box thinker is probably
something of an understatement, and it is great
to have people working with Asterisk building the
types of tools that Jay is developing. Jay and I
have talked quite a bit with each other on
subjects regarding Asterisk and Adhearsion, and I
recall an article on Adhearsion was written
by Jay on the dock of my weekend place. I hope
that we can continue this discussion as a
conversation of ideas regarding how Asterisk can
continue to evolve, even if we disagree on some
of the notions that each of us bring to the
table. While I may not entirely understand some
of Jay’s issues he outlines in his posting,
perhaps my comments on some of what I believe
are his points will help us both better
understand where improvements or clarifications
can be made.
I’m going to try to address Jay’s blog point by
point, but it’s made complicated by Jay’s having
mixed so many business metaphors and by leaving
terms like “market” unspecified or variable.
Also, Jay, like my mother, seems to have issues
differentiating between “fact” and “his personal
opinion”. Much as mom thinks I should like
salmon because “it’s good”, Jay makes a number of
assertions which are simply his opinion, but he
states them as fact. I’ll try to point some of
these out, and where possible I will try to use
actual metrics to either support or refute those
claims, and where not possible, some research
would be required to be able to support them.
Lets start with Jay’s largest “one thing” he
wants us to understand, that “Asterisk Has Not
Crossed the Chasm Yet”. The first, most
fundamental problem with trying to address such a
statement is that Jay hasn’t actually chosen a
market. It’s much easier to try to decide if the
“iPhone” has crossed the chasm or not because its
purpose is generally clear. It’s a phone, and
the market is consumers. But what exactly *is*
Asterisk? Most people think of Asterisk as a
PBX, even though it is a great deal more today.
Is he saying that it has not crossed the chasm as
a PBX? Jay is particularly interested in
Asterisk as a platform underlying his technology
for developing applications. Is it that Asterisk
has not crossed the chasm of being the defacto
development platform of choice for IVR
applications? For conferencing?
Jay says he feels like Digium has been caught up
in the misconception of the growth of the
visionary market. In reality though, Digium’s
primary market is not Jay’s market, because
Digium’s *commercial* focus is primarily on the
SMB and to some degree Enterprise PBX market –
almost certainly the largest *single market*
telecom products that you can build with
Asterisk. Digium’s *primary* business is not to
sell to developers like Jay, it’s to sell to
customers like Joe’s Laundry down the street,
through resellers and channels that almost all
fit in the early and late majority category.
Digium’s work on expanding Switchvox and channels
clearly indicates that we *clearly* understand
the importance of that market and precisely *are*
doing the right things to cross the chasm in the
PBX market. We understand that the early and
late majority customers are looking for a PBX
product that is feature rich, inexpensive and
easy to use, and that’s precisely what we’re
building. You’re not going to be able to take
regular open source Asterisk without any
packaging and try to take it to your average end
user. It needs to look, feel and smell like
something they can actually use, and doing so
demands the creation of a derivative work focused
on the market you’re going for, and *that’s* what
you’re going to cross the chasm with. Jay says
that Asterisk isn’t sticky because it’s too hard
for an end user to use — and for many end users
that may be true, but that’s not the product that
*should* be bringing them — it’s AsteriskNOW or
Switchvox or some other, more cooked product,
that is the right thing if you want a PBX.
That’s like saying that Chevy 350 small block is
a lousy product because most people don’t use it.
It’s not a bad product, it’s just that people
want cars, not just motors.
Having said that though, the direct market for
Asterisk, as a project, is of course not early
and late majority PBX customers — it’s
developers who want to build products that
customers in turn will actually use. I *think*
this is the “market” that Jay is actually
addressing with his “Otto” example, and it is a
market Digium is trying to pursue for Asterisk as
the benevolent maintainer of the software — not
as its primary commercial interest (although
don’t get me wrong, we’re happy to sell our wares
to developers as well!) Clearly if he was a
traditional PBX customer without technical
background, downloading Asterisk and building /
installing it on his own newly installed Linux is
not likely the best place for him to start. If
Otto is a developer, then that’s probably a good
way for him start (although AsteriskNOW might be
a little better place to start). Jay calls
asterisk “prohibitively unintuitive” and if
you’ve never used Linux or a command line, that
may be true, but if you’re a developer, you’ve
configured tons of things before, and something
like AsteriskGUI can get you off to a running
start from which you can start fooling around
with some of the developer technologies, although
currently the best single source of those tools
may be Google.
Jay drives his opinion that “the average amount
of time a developer plays with Asterisk is less
than one week” and that “the developer retention
rate of Asterisk is less than 1%” but in spite of
being in bold print, these are not statistics but
merely his opinion. To my knowledge, no one has
ever really researched this, and I’m uncertain
even anecdotal evidence of “sticky” numbers of
developers is significantly different than any
other major project that sees experimental use by
the overall OSS developer community. In any
case, Jay’s premise seems to be that Asterisk is
simply too difficult for a developer to build
from. Fortunately for us (and unfortunately for
his argument), such a premise is largely in
conflict with reality. For example, lets look
just at a series of recent projects built at the
Interactive Telecommunications Program (ITP) at
New York University:
* Botanicalls - Your plants call you when they’re
low on water. Each plant has its own personality
based on its species. Further development is to
include other environmental factors.
* I-PLATE-U - Record and retrieve messages based
on someones license plate. Heavily uses speech
recognition to avoid having to type on the
keyboard
* Popularity Dialer - Calls you and plays one of
5 different pre-recorded “half conversations”
including the “Cousin needs help” call, to help
you get out of a sticky situation and the
original “popularity” call where your friend
calls trying to get you to join your other
friends, but you decline because you’re on a
wonderful date.
* Cause Caller - Automates the process of
allowing you to call in to politicians in support
of various political causes.
* Booty Dialer - Automatically calls each entry
on your “booty call” list until someone responds
positively to the interrogatory, at which point
your phone rings and you are connected with them
directly, saving lots of time.
* Canal Street Station - Interactive telephony
mystery which is played out by calling a toll
free number from various public pay phones in the
canal street station
* Rap Happy - Allows you to use your phone to
record rap music (presumably other music) via
your phone.
Now, obviously these are extremely creative
applications of Asterisk, but what is perhaps
most interesting about these is that the ITP
program is not an engineering program — nor are
the creators of these applications engineers –
they’re artists, in the art department. This
program has taught them to use Asterisk as a
medium for creating telephonic art, or at least
taught them how to accomplish telephony tasks
without having a CS degree or years of
development and sysadmin experience. Of course
there are lots of other business examples that
actually generate revenue, but the point is that
lots of people make real applications with
Asterisk today, even in its current form.
So, do I think there’s no truth to anything Jay
is suggesting on the API? First of all, like
Jay, both Digium and myself want to see the
advancement of the technology worldwide — that’s
why we invest so heavily in Asterisk. How about
this for a fact and — the largest single
engineering team at Digium is the team dedicated
to Open Source Asterisk development. Certainly I
agree there are improvements that can be made.
Asterisk has at least three different ways to
“program” it, including the manager API, the AGI
interface and the various extensions languages
like AEL and LUA. Like Jay says, Asterisk and
open source telephony are virtually synonymous
because Asterisk has been around for such a long
time that it is able to talk to virtually every
VoIP and TDM telephony technology. It has
virtually every core traditional telephony
application (like voicemail, call queues, ivr,
conferencing, etc) already integrated and has the
maturity and stability to be relied upon by a
wide variety of OEMs, enterprises and businesses.
What Jay, Digium and I all want to do is to make
those technologies as accessible as possible to
the widest variety of developers who are going to
take this core out to new places and create more
applications that haven’t existed before.
Jay proceeds to enumerate several specific
concerns about Asterisk which again I’ll try to
address. Firstly, he mentions that if you build
an Asterisk system, you have to be a developer.
This is no more true of Asterisk than it is of
Apache or even SSHD for that matter. Each of
those applications has various configuration
files that control its behavior. In each case,
you can either use a graphical interface or you
can choose to configure the system by manually
editing the files. That’s a matter of personal
preference. By Jay’s suggestion, anyone editing
a config file is a developer. There is probably
a continuum between developer and user wherein as
a system becomes more flexible, and more options
are presented to the end user, their need for
understanding and abstracting concepts becomes
higher. This is precisely why Asterisk is a
product for developers, and products like
AsteriskNOW or Switchvox are for end users, and
simply utilize Asterisk as the functionality
provider. I know Jay is annoyed that
extensions.conf is Turing equivalent, but hey,
that’s not how it was intended to be used, and if
you choose to use that even when there are plenty
of other, easier to understand, easier to read
ways of configuring Asterisk, that’s your
prerogative just like you can choose to write in
assembly language if you wanted to. Just because
it’s there doesn’t mean you have to use it that
way. What extensions.conf *is* useful for is
listing a bunch of extensions — precisely its
original purpose. And if you don’t want to use
it, use the Asterisk GUI. Although Jay describes
this as “irrefutable” evidence that configuring
Asterisk makes you a developer, it no more makes
you a developer to configure Asterisk than it
does to use the bash command line, which by the
way is substantially more complex, also is Turing
complete, and people also use primarily for the
purpose of simple things (launching applications
for example) and not sophisticated software
development.
Jay also seems to be concerned that Asterisk is
written in C. One of Asterisk’s core missions is
to scale and be efficient. Writing in a language
like C that supports that goal is important to
allowing these creative apps that live on top of
Asterisk to be able to scale to the number of
users that end up wanting to actually use them.
Yes, programming in C requires you to be careful
in your programming, but I think it’s worth the
extra effort to write a good program once to
allow the end user fantastically better
scalability in the long run.
Leveraging Asterisk beyond just being a PBX is
typically done at different levels depending on
the sophistication of what is desired. If you’re
just doing a basic IVR, you can largely do that
within AEL (or heaven forbid, extensions.conf)
using many of the applications and variables
included within Asterisk. Granted, this is
learning another language, but it’s a language
very specifically designed for operating within a
switch, and its efficiency is tremendous because
it’s so tightly coupled to the operation of the
system. Beyond the basic IVR’s, people start
wanting to use a full-fledged general purpose
programming language. Which language to use,
however, is clearly not an area of agreement, as
evidenced by the AGI and AMI bindings for Perl,
Java, PHP, C, .NET, and many moreŠ and of course
Adhearsion for Ruby. One important thing that I
think we can try to do within Asterisk is to make
primitives that more easily support binding of
other languages. Jay talks a lot about framework
– and framework is a great concept — but there
are lots of languages out there, and they largely
each have different frameworks and programming
paradigms about them, and each attract different
sorts of developers and lend themselves to
different ideal applications.
Asterisk, as with most telephony software,
suffers from misperceptions about goals, and
there are complaints no matter which side of the
double-edged sword is being swung. “It’s too
hard to configure!” is the cry that I hear from
this post. But equally loud is the group who
yells “I want to control all the details.” We
agree that Asterisk is complex; but telephony is
complex in the details. This is not dissimilar
to Linux, firewalls, routers, mobile phones,
Apache, databases, or a host of other complex
systems which have various and sometimes
competing layers of control put in place over
them to simplify adoption and integration.
It is possible to abstract complex sets of
primitive elements once you have a full
collection of those primitive elements. It is
*not* possible to go the other way around.
Asterisk (the project) is working towards a
(more) complete set of basic functionality which
allows a developer to achieve as many telephony
tasks as possible, even when some of those tasks
may be complex to develop if you are creating
them from scratch. When we say “developer” in
this instance, it could mean either someone
directly building an application, or someone
building an API into another programming
environment. We are hoping and encouraging
projects like Adhearsion, FreePBX, the various
language libraries, and others to create
abstraction layers and configuration layers on
top of the Asterisk toolset in order to
accomplish specific goals which may be far
outside our ability to comprehensively predict or
develop. That is what we envision as our path
towards the “Tipping Point” for application
developers.
Jay continues by trying to describe a view of how
an Asterisk System and an Asterisk Application
are different. If you get at the core of the
concern though, it’s that he thinks people build
Asterisk systems in ways that aren’t the ways he
wants to do them. He says they’re built the way
they are because of deficiencies in the
extensions.conf configuration format. Only
problem with that logic is that if the
extensions.conf format was deficient, they would
obviously have used one of the other wide variety
of programming techniques available (maybe even
Adhearsion). If someone chooses to use AEL, or
even extensions.conf to create an application of
Asterisk, it is precisely because they *were*
able to do it with those systems that they chose
to do so, not because they couldn’t — again,
having those systems doesn’t take away from your
ability to use any one of the native languages
and frameworks that are available for building
applications on Asterisk. Said another way:
Don’t complain about what languages people use
(even if you want to call extensions.conf a
language). Use what you like best, and let
people choose what language they want to use!
There certainly are a lot of them! Asterisk is
*already* being used as an underlying layer by a
huge number of abstraction libraries - and we’re
hoping to continue that trend to make programming
telephony applications easier by developers in
their own environments. You don’t need to be a
dialplan wizard to write applications that use
Asterisk: “There’s more than one way to do it.”
Now, another point that Jay makes that I think is
valid (and one Digium is trying to address) has
to do with how you install additional features
and applications in an Asterisk-based
environment. The Asterisk Marketplace is
precisely intended to be a place to allow people
to have applications more readily distributed,
including through graphical interfaces, to be
installed not just on Asterisk but on PBX
products like AsteriskNOW and Switchvox. Sorry
to be vague on this one; stay tuned.
To continue on: I think the “The Long Tail”
illustration cited could not be more perfect. If
you look at the green portion of the curve, that
is where Digium’s primary business focus is
placed, and if you look at the yellow portion,
that’s where Digium’s primary focus for Asterisk
is placed. The former is best address by a
product and a company, and the latter by a
project (in this case sponsored by a company),
where we enable lots of small micro-markets that
can be attacked by a variety of developers
enabled by technologies like Asterisk either
directly or through interfaces like Adhearsion
and others. We actually agree on this point-
there are more end users (ultimately) on the long
tail, even though that long tail is very long and
thin. Asterisk, the project, is doing all it can
to help with building apps out on that long tail
by creating a free toolset that developers can
use to address their infinite niche markets.
This “long tail” will exist because of tools like
Asterisk, not the other way around. Digium, the
company, is creating a set of products based on
or around Asterisk that address the head of the
market, which by far is the largest single niche
in the diagram by far. One end of the diagram
supports the other - the products that Digium
creates around the largest and leftmost portion
of the curve creates and supports the tools that
will enable the proliferation of independent
applications that fill the “tail” on the right.
After reviewing this article a few times, I am
more unclear on what the issues are that Jay is
actually raising. This perhaps is due to my own
myopia when looking closely at Asterisk and also
having a pre-conceived notion of what I feel are
the shortcomings of the project today. As a
possible continuance of this discussion, perhaps
a succinct list of problem statements would help
me in refocusing what we need to do to solve what
are obviously to Jay issues of concern.
Certainly I wouldn’t claim Asterisk is perfect,
but the developers in the community and within
Digium are interested in hearing the things that
are necessary, and then as they are needed, the
code can be written to fit the solution. I
appreciate that the requirements are changing as
Asterisk becomes more mature - we’ll try to stay
on or ahead of the curve based on the
perceptions, requests, and most importantly, the
work and contributions of the community.
Reading between the lines in this post, it’s
clear that Jay has what he believes is a great
idea with Adhearsion. I don’t disagree - it’s a
very neat solution for people working with Ruby
On Rails, and simplifies life for developers in
that particular arena. Perhaps working so
closely with high-level abstractions makes the
details of telephony a chore - this is
understandable, but I believe that additional
tools have been and will be built that allow us
to cross “the chasm” for both developers and end
users - I think we’re crossing that chasm
already. Perhaps after waiting a few months it
will be the case that Jay’s company will
announce a more commercial solution to the set of
problems he ambiguously identifies today; if so,
I would caution him not to become myopic in his
own way when identifying the uniqueness of his
own solution and how it resolves perceived
shortcomings of Asterisk. We’re happy to see
more interest building around the Asterisk
ecosystem, and we hope to help Adhearsion and
other firms build on Asterisk as the engine for
their telephony applications.
In summary, I’d say that Jay and I share the same
goal of making Asterisk a more accessible
technology both for developers and for end users,
and I’ve invited his participation in the future
of Asterisk at the next Astricon 2008 (and
subsequent developer meeting), which he has
eagerly accepted to particpate in! I look
forward as always to working with him.
Mark
Mark Spencer
13 Aug 08 at 10:02 pm
Astricon Cage Match: Jay and Mark mano-a-mano! Get your tickets now!!
Hey, both of you guys are heros of mine so I’m not taking sides. You guys are looking at different parts of the elephant. Asterisk is great at what it does, but it’s not all about Asterisk. The internet runs on Apache, but there’s little point in making Ajax a module for Apache. If you do, you end up with things like Booty Call as an example of what you can do. That’s fun, but it doesn’t pay the rent. Asterisk can pay the rent in the SMB world, Adhearion can pay the rent in the ‘creative class’ world.
Don’t go all Vista with Asterisk - make it a rock solid back end without bloat that is easy to hook into for development with independent tools. And don’t piss on Asterisk for not being the perfect solution for things it isn’t trying to do.
Jockey
14 Aug 08 at 7:57 am
Go compare Freeswitch to Asterisk. Much better design.
Jason
14 Aug 08 at 2:39 pm
Bundling up something terribly complex into something simple to work with is everyone’s goal I think. It looks like we have identified the problem so it’s time to work on the solution.
anthm
15 Aug 08 at 2:08 pm
I highly recommend FreeSWITCH to everyone that is interested in VoIP and applications.
You can do nice things like calling scripts from the extensions in the XML dialplan, there is also YAML support through mod_yaml.
There is also socket support (mod_event_socket), xml-rpc, I believe they are also working on JSON/REST support for writing web apps on top of it.
You can also abstract the low level functionality into a high level fashion using scripting languages or XML/YAML, etc.
FS rocks.
Diego Viola
16 Aug 08 at 10:11 pm
Thank you for the good read and the subsequent comments, specially of Mark’s. We don’t get to read him much these days. My work on Asterisk has been largely on conferencing integration and remote monitoring of non commercial, non public projects. The good thing about Asterisk is that once you start working with it, it does not let you stop. As Mark mentioned, there are many things that you could do with Asterisk. I feel sorry for Otto because he is not willing to go beyond the fence. Perhaps it is because lack of people to guide people like Otto and inspire them. I hate writing unless it is really needed but you and Mark got me writing something at 2:00 AM while waiting to get somewhere. I call that inspiration (You getting me to write!). Hope your article and the comments will open some eyes and and make some Ottos see Asterisk for what it is.
Ravenii
19 Aug 08 at 2:36 am
I also read your not meant to be published post (Sorry my reader, newsgator picked it up). That is even better!
Ravenii
19 Aug 08 at 2:55 am
How to get more Asterisk developers?
Here is a solution: make it easy for web hosting companies to host PBX applications as it is easy to host web applications.
The phone model is different from the Internet model from a business point of view: you can get money from every “page hit”. So hosting PBX applications can be a good businness. And if you make it easy for web developpers to integrate a phone (audio) interface into their web application you will give them an additional way to make money.
So we need easy Asterisk hosting and framework integration with web applications !
Olivier Mengué
19 Aug 08 at 10:28 am
If I may paraphrase :
What you’re saying is that Asterisk is the sendmail of telephony.
For those who don’t understand - sendmail comes with a legendarily complex config file, and sendmail’s developers also completely missed the point that they were asking their users to be programmers, and not just that, but to program in a terrible language.
Noel Grandin
19 Aug 08 at 11:21 pm
The problem with Asterisk, is Asterisk itself. There’s just so many things wrong with it, it just can’t be salvaged, it’s design is flawed all the way to the core.
Callweaver has attempted to improve the situation by replacing a number of the key flawed components with either implementations leveraging some other solid open source project, or by rewriting when necessary. This helps a lot, but at the core, it’s still the same flawed Asterisk design.
I have a number of both Asterisk and Callweaver deployments under my belt, and certainly a few years ago, Asterisk was the only option. I have no regrets having used it, it has allowed me to do things in telephony I had never imagined would have been possible a decade ago. In fact, I would have probably never got involved in telephony had it not been for Asterisk. It has served it’s purpose and served it while (despite all it’s flaws).
However, for now, the future of IP telephony in my opinion, no longer lies with Asterisk or it’s derivatives. FreeSWITCH is the future.
FreeSWITCH is a younger project obviously, and hence there’s a few things that might be easy to do in Asterisk, yet a little bit more cumbersome than one would want in FS, but there’s so many things that FS does better than Asterisk, it blows my mind. In addition, there’s so many things that you can do with FS that you’ll never be able to do with Asterisk (at least not in any way as clean and elegant, and without having to write a C module). FS is well designed and it doesn’t get in the way. It really is a pleasure to use and so powerful at the same time. It’s also fairly well documented (by the project itself).
That said, you pretty much need to be a programmer to make good use of FS. Certain things that could be done in Asterisk in extensions.[conf|ael] are going to require writing a few lines of code (and I mean, Javascript, Perl, Python, or one of the other languages FS has some support for), but that’s actually part of the beauty of FS. Other things that required a convoluted mess to achieve in Asterisk are often achievable in some simple, clean and elegant way in FS.
That said, FS, due it’s design will certainly yield some PBX projects built on top of it that will have a much better chance at offering a friendly, yet powerful PBX offering. Things LaTeX on top of TeX if you will.
Yves
24 Aug 08 at 4:14 pm