This morning I was looking for a way to handle incoming email in a web
application (similar to the way Highrise and Evernote let you email
things to a special email address and have them put into their system).
There are a number of ways to do this via procmail, or by using
something to connect to your mail server using POP or IMAP and reading
emails, but I was looking for a way to do this without having to host my
own email infrastructure. Ideally, I want something like
Twilio, that will receive the email and then do an
HTTP POST to the endpoint of my choosing.
Here’s what I found.
Still in beta (and free while it is), this looks robust. It’s also
available as a Heroku addon, if that’s how
you roll.
(A tip of the hat to @peterc for pointing
me to this one)
Looks similar to CloudMailIn, though not in beta. There’s a free plan
for up to 100 emails a day, and then it goes up from there. Their site
was down when I first went to it this morning, which makes me a tad
nervous, but that may well be an isolated thing.
SendGrid
SendGrid is a heavy hitter in the email space, mostly doing outbound
delivery. They do however have a Parse API that seems to perform the
same function as the other two services. I’m not sure on the pricing
here, their basic plan is $9.95 per month for 10,000 emails, but I’m
not sure if that includes incoming or not. UPDATE: I heard from
SendGrid. Their plans cover both incoming and outgoing, so for the in
the case of the $9.95 plan, it could be a mix of both, up to 10,000
(thanks to Twilio’s @johnsheehan for
the pointer to SendGrid)
I haven’t used any of these yet, so I can’t make an endorsement of one
over the other, but I thought I’d post it here in case anyone else is
looking for this kind of provider. If you have experience with any of
these, please comment with your opinion.
We are headed toward a time where the workings of government are much more visible to the American public.
Through things like the Freedom of Information
Act,
this information has technically been available for some time - but
not in a form that is easily consumed. This is starting to change.
The emergence of open APIs that provide access to information about how
the government is operating is a massive step in the right direction. It
will, I hope, bring forth a new wave of websites that mine the data that
these web services provide, and expose it to the world. Voting records,
government expenditures, bids, and bill details all need to be made
available for anyone to consume.
Here are a few of the APIs that I have come across. Some are available
from the government themselves, others are from third parties.
This API, part of a growing set of APIs from the
Times, let’s you program
Congress. Well, not exactly:
The initial release exposes four types of data: a list of members for
a given Congress and chamber, details of a specific roll-call vote,
biographical and role information about a specific member of Congress,
and a member’s most recent positions on roll-call votes.
…
Our database contains House votes since 1991 and Senate votes since
1989. House members are from 1983 and Senate members date back to 1947
This API provides information about campaign contributions for
state-level campaigns.
The Sunlight Labs API provides methods for obtaining basic information
on Members of Congress, legislator IDs used by various websites, and
lookups between places and the politicians that represent them. The
primary purpose of the API is to facilitate mashups involving
politicians and the various other APIs that are out there.
This site, another project of the Sunlight
Foundation tracks word frequency from
the congressional record. Most frequently used word: Proposed.
One of the few official government APIs, this allows you to find out
where your money goes:
Have you ever wanted to find more information on government spending?
Have you ever wondered where federal contracting dollars and grant
awards go? Or perhaps you would just like to know, as a citizen, what
the government is really doing with your money. The Federal Funding
Accountability and Transparency Act of 2006 (Transparency Act)
requires a single searchable website, accessible by the public for
free that includes for each Federal award:\
- The name of the entity receiving the award;\
- The amount of the award;\
- Information on the award including transaction type, funding
agency, etc;\
- The location of the entity receiving the award;\
- A unique identifier of the entity receiving the award.
The Republicans are ahead of the Democrats on this one, but I would
doubt we’ll have to wait to long to see them follow suit. This site is
similar in concept to the Times' Congress API mentioned above (though
obviously only for the Republican members), but provides some additional
information that the Times does not.
Conclusion
This is only a sampling of the APIs that are available, and hopefully
this is only the beginning. I’m optimistic that both the government and
third parties will provide more of these, and that groups will make use
of this information to expose the inner workings of the government to
the people who elect them.
Are their any APIs that I missed? Are there any sites that are using
this information in interesting ways? What do you want the government to
make easily consumable that it isn’t today?
In my time as a developer, and now managing a team of developers, I have
come to realize that there are two kinds of programmers: the Journeyman
and the Craftsman. These terms aren’t mine - I’ve seen them used other
places - but they describe the developers I’ve worked with pretty well.
The Journeyman
…knows one programming language.
…knows one operating system.
…can’t be bothered to learn something on their own.
…doesn’t know anything about the operating system or hardware their
applications run on: “Someone else takes care of that”.
…never masters his tools. “I know my way around my IDE, that’s good
enough”
…doesn’t refactor: “It’s ugly, but it works. Leave it alone!”
…only learns about the part of the system they are working on. No need
to learn the rest of the system: “That’s not my job”.
…doesn’t want to take on an unfamiliar technology: “I haven’t had any
training on x”.
The Craftsman
…knows a handful of programming languages, and is always on the
lookout for the next one he should learn. He knows that learning any new
language will stretch his mind and make him a better programmer in the
language he uses day to day.
…devotes time to learning about new technologies, and helps to make
others aware of them.
…understands the platform and operating system his applications run
on, because he knows that’s the only way to diagnose many problems.
…masters his tools. He can perform magic in his chosen editor, and is
always looking for ways to make himself more efficient.
…rarely passes up an opportunity to broaden his knowledge of the
system he is working on.
…is always willing to take on something he’s unfamiliar with. He can
pick up most things pretty easily, and enjoys the challenge of learning
something new.
One craftsman is worth three or four journeymen. Easily.
It’s the journeymen whose jobs often end up moving overseas (and
rightfully so, they add little, if any, value).
The longer I manage development projects, the more I value the craftsmen
I have around me.
This is the most
concise representation of why web standards are important that I’ve
seen.
Wil Shipley is the author of several well known Mac applications,
notably
OmniGraffle and
Delicious Library , and has a
well-read blog. In a post
here
he asserts that “Unit testing is teh suck”.
Although Wil is a bright
guy and
very likely a better programmer than I will ever be, this is completely
out of touch with reality. I suspect anyone that has done
TDD, or even
used a unit testing tool would agree with me.
Wil starts off with this statement:
I've NEVER, EVER seen a structured test program that (a) didn't take
like 100 man-hours of setup time, (b) didn't suck down a ton of
engineering resources, and © actually found any particularly relevant
bugs.
Well, I think Wil may be exaggerating a bit with the 100 hours nonsense,
but let’s assume he’s not for just a second. I’ve never written a single
line of ObjectiveC code, but if
OCUnit is anything
like the other unit testing tools out there (NUnit, JUnit, Test::Unit),
this is a ridiculous statement. As for not finding relevant bugs, I find
it unlikely as well.
Next up is this statement:
It's actually provably impossible to test your program with every
conceivable type of input programmatically, but if you test by hand you
can change the input in ways that you, the programmer, know might be
prone to error.
Sorry, wrong answer. Anything you can do
manually can (and should) be automated. Two reasons:
- Manual tests aren’t reproducible. Duh.
- Manual tests take a lot of time to do. Because they’re manual.
Double-duh. Furthermore, things that are time consuming are often
skipped when people get tired, lazy, or just need to get the thing
out the door. And unless you wrote down all of those manual tests,
if the person who wrote the code (and was doing the testing) leaves
the company, you’re in trouble.
Summary: Unless you have a good set of unit tests, real regression
testing is nearly impossible.
Next, Wil describes his testing methodology:
When you modify your program, test it yourself. Your goal should be to
break it, NOT to verify your code. That is, you should turn your huge
intellect to "if I hated this code, how could I break it" as SOON as you
get a working build, and you should document all the ways you break it.
\[Sure, maybe you don't want to bother fixing the bug where if you enter
20,000 lines of text into the "item description" your program gets slow.
But you should test it, document that there is a problem, and then move
on.\] You KNOW that if you hated someone and it was your job to break
their program, you could find some way to do it. Do it to every change
you make.
There amount of time it would take to do this on a program of any size
is ridiculous. Although Wil may have the self-discipline to do this, the
rest of us likely don’t. And again, it’s not a repeatable process.
Here’s the bottom line: Unit tests are important. Extremely important.
And they don’t add time to a project, in fact in most cases they will
reduce it. At first blush this doesn’t make much sense, but here’s the
rationale:
- You typically write this code anyway, but end up throwing it away.
How often have you written some chunk of code, then written a piece
of throwaway code to test it. Unit tests are all about taking that
code and making it automatic.
- You’re trading test-coding time for testing and debugging time. Yes,
it takes time to write unit tests. But it also takes a lot of time
(arguably more) to run a bunch of manual tests. It also takes time
to debug problems found later on that could have been
caught earlier.
Wil has forgotten one of the basic rules of software development: Catch
Bugs Early. If you catch a bug when you’re just getting started, it
costs nearly nothing to fix. Find that same bug after you have released
it to the world, and it costs significantly more - particularly if
you’re not talking about a web application.
On top of all of this, there are a couple of other points worth making
about Unit Tests:
- Refactoring, or any type of major code change, is scary without good
unit tests. Often, people avoid changing the code altogether and
just live with it’s warts. I can’t find the post now, but one of the
37Signals guys blogged about how having
unit tests made the architectural changes that were made to Basecamp
a lot easier.
- When you have to integrate with other developers, unit tests
are invaluable. This is particularly true when coupled with
Continuous Integration.
- Unit tests (if done right) will improve the architecture of
your code. You simply can’t have a highly-coupled application if you
are unit testing everything independently.
- Unit tests serve as documentation for how the code
actually works. Forget that dusty API reference that hasn’t
been updated for three versions. A set of passing unit tests tells
you what the code does right now. Paired with a code coverage tool,
it can even tell you what’s not documented.
Summary: Unit tests == Awesome, and represent all that is good in the
world. I certainly don’t think that they’re a panacea, but they’re
certainly not “teh suck”.
I didn’t expect this to turn into such a rant, but when people who are
respected and admired in the community make harmful and incorrect
statements, they need to be corrected.
Someone wrote a
programming
language consisting entirely of whitespace (you knew it would happen
eventually). Now maybe people will stop making fun of Python.
via
File this under “Useful things to remember”.
The following bit of command line magic will export the schema of your
database.
mysqldump –no-data –tables -u
YOUR_USER_NAME -p DATABASE_YOU_WANT_SCRIPTED >> FILENAME.sql
An example looks like this:
mysqldump –no-data –tables -u larry -p contacts >>
contacts.sql
If you want the data as well, omit the “–no-data” portion.
The Backpack
web service has been available for a few short hours, and already
O’Reilly has started using it on their
Radar
Blog. They’ve devised a nifty way of tying together content on their
public backpack pages, with the Amazon API in order to produce their
books
page. This reminds me a lot of shell scripting on a *nix system, tying
together applications to build something new.
Very cool stuff.
