Amazon pub/sub in the cloud

Amazon keeps extending its cloud offering. They have just added Amazon Simple Notification Service (SNS). SNS is a publish/subscribe mechanism.

Integration-As-A-Service
As explained in earlier posts, I expect Integration-As-A-Service to become more important. One of the larger players (Amazon, Google, EMC, Cisco, Microsoft, ...) may one day come up with a wonderful solution for Business-2-Business communication between organizations.

When I first learned about Simple Queuing Service of Amazon back in 2006, I intially thought that SQS could serve as a transport mechanism for B2B communication. But that didn't work out. As the message size of SQS was very limited, data first had to be stored on S3. Authentication and authorization were also very limited.

So I looked around in the SNS documentation to see what SNS actually is and see if it can serve as a basis for B2B communication. Amazon thinks SNS is usable for B2B or application integration:

Application integration: Amazon SNS can be used in workflow systems to relay events among distributed computer applications, move data between data stores, or update records in business systems. For example, in an order processing application, notification messages may be sent whenever a transaction occurs; a customer places an order, the transaction is forwarded to a payment processor for approval, and an order confirmation message is published to an Amazon SNS topic.

Some facts

• Messages can be published over HTTP, HTTPS, E-mail or SQS
• Proprietary solution/mechanism, not based on any standard (no AS1, AS2, SFTP, WS-Notification, WS-Eventing, ...)
  
• Messages are (again) limited to 8KB. Just like SQS: too small.
• Authentication is based on AWS accounts, so also every subscriber requires an AWS account, hindering factor.
• Messages are pushed, not polled. This is good for performance. For polling, use SQS.
  
• But when pushing, the subscriber must expose a web service or mail account. How to secure this: no authentication from Amazon to endpoint receiving notifications; no basic auth, no support for client certs, ...
  
• Messages are signed by Amazon. This is good, very good. Signing is based on HmacSHA256.
  

Conclusion:
Nice and interesting, but not good enough... In particular the message size remains a blocking factor.

Questions left:

• What happens if messages cannot be delivered for a longer periode of time? E.g. when a subscriber disappears?
• How does a message that is published over HTTP exactly look like (signed, JSON)? What parameters are passed in the URL?
  
• Can an SSL endpoint with self-signed cert receive notifications?
• What if SSL cert of endpoint is expired?
• Are mail messages signed and if yes, how?
• How and when are messages actually persisted?
• The publish service isn't idempotent it seems?
  

PS: all based on reading the docs, must confess that I didn't actually test it

Authored by: Guy