Archive for the ‘database’ Category
I needed a way to be able to shamelessly plug the posts I recently bundled into the booklet “Write Something” again.In a similar way to the last time I did it in Proof of Concept: Google Docs Mail Merge Form
As I explained before I have set up a system to automatically mail somebody when the enter their address in the form, my issue was that I wanted to add a unique blogpost which they would only be able to get by signing up. Naturally I wanted to style it in the same way the posts are styled in this blog. Again I turned to the documentation, specifically the Class MailApp which I was using to send the mail. Using this documentation I had a starting point. I wanted three changes to the current script:
- keep the plain text
- add HTML message
- add inline images
I needed a way to be able to shamelessly plug the posts I recently bundled into the booklet “Write Something“. I want to build a list, and offering something which adds value for the subscriber is a good way to do this. There is a host of good material which you can use to help, so I won’t elaborate on that in this post.
I have a hosted WordPress.com blog, which means that I can’t run a local script to collect the mail addresses and mail them, so I turned to Google Docs’ Form functionality for the entry form, naturally I give them the option to download the booklet there, and I wanted to send the subscriber a message to thank them. In the Google tutorial: Simple Mail Merge they explain how to do a mail merge using the Script Editor. I wanted to go a little further and have it send a mail with thank you note and a link to each subscriber as soon as they filled in the form.
Having worked without RDBMS for much of the beginning of my carrier I have always been confused by people’s love of relational databases, in my mind they are merely a collection of CSV files with relationships, with some of extended capabilities that all other databases have such as indices and caching. I love that the concept of something that is not a Relational Database, or a complicated Key-Value store, has found it’s place in the world and it’s called NoSQL.
And didn’t we already have a solution which matched the requirements: scalable, ordered, hierarchical, sharded, consistent, atomic, distributed and object? And wasn’t it a key-value and document database with graph capabilities? An engineer wisely said: “Relational databases give you too much. They force you to twist your object data to fit a RDBMS.” What system doesn’t force you to twist your object data and still allows you to maintain the objects in the way you desire?
When we faced this issue we were having much trouble with a traditional database vendor and the mail software they were producing, we wanted to extend the capabilities of this software and not be reliant on the on-disk mailstores they provided. Mail should be stored distributed and be approachable from different angles, whether it is with a traditional POP3 client – the norm; a HTTP browser – emerging; or a IMAP4 client – which in those days was hideously complicated as the RFC had some features which were almost impossible to implement easily. We also wanted to be able to add USENET – which had the same format which we also wanted to be able to store, and even chat – be it IRC or private messaging. And while we were at it we might as well add FTP in the mix.
The external connections would be implemented in an Enterprise Service Bus design pattern, and the storage part was what posed the real problem. All of the data would need to be secure, distributed and/or sharded over multiple locations for efficiency and security. And with security as our first demand we thought of an open standard which we and almost the whole planet used and uses for their internal authentication. A database which has a key-value store at the core, based on a protocol extension written in round about 1993 and optimized in 1996. A database idea so SMART that every huge large software company in the world sells it: LDAP.
“LDAP?” I hear you cry, “That’s No NoSQL!” And you’d be right!
Image source: LinkedIn NoSQL Group
I was reading ESB in Action* while preparing to implement Apache ServiceMix and was interested to read about JiBX, which is a tool for binding XML data to POJOs (Plain Old Java Objects). What it immediately reminded me of was Hibernate, which does ORM (Object Relational Mapping), and the possibilities there were for integration. This brought the thought to my head that together they could be used to create a WebService which could expose features of the database easily.
A search brought me the view of a Hibernate developer who pointed out in 2004 that there was a “impedance mismatches here: object/relational and object/hierarchical.” And I believe that would be true if it wasn’t for the fact that much of the data in relational databases is mostly hierarchical in nature. Even the core pattern of the embedded indexing in Hibernate Search assumes that the data being indexed is a nested hierarchy or inclusion hierarchy. I won’t get started on the issues that Hibernate Search has due to this impedance mismatch, needless to say there are a number.
The JiXB documentation is quite clear, and makes it easy to implement.
Since I started working for my company I’ve been exposed to PCI DSS (Payment Card Industry Data Security Standard), “It was developed by the major credit card companies as a guideline to help organizations that process card payments prevent credit card fraud, hacking and various other security issues.“1 There are only a small amount of requirements that need to be implemented, although these can be pretty substantial for some customers. I can also be difficult to understand the details of these 12 requirements for compliance.2
Being a programmer by nature I’ve often been told that the nuts and bolts of what I do, the part I enjoy, are a little complex. PCI is something different, everybody can understand that credit card data needs to be protected from unauthorized access. Not just credit card data, but all data that could potentially be used in identity theft. Which means that a policy or control needs to be implemented to control this, and note any non-compliance.
PCI is just about protecting your “Cardholder Data“:
I know first hand that most of the banks in the Netherlands, and in most of the world, are quick to discover credit card fraud. They are also quick to payout and correct the issue for the customer, because the chance that customers will loose faith in the bank is high if they don’t. Yet ultimately these customers are still paying for all the fraud committed with all the credit cards. Banks, payment service providers or retail merchants, who have your Cardholder Data, have all the data needed for this kind of financial identity theft and fraud, and more…
It may seem obvious that this data is stored securely, credit card use is ubiquitous. Yet the large banks have had the same problems with data leakage as small retailers, which means the data must be secured from the customer right to the bank who finally processes the payment to avoid this type of leakage. The problem is that payment service providers or merchants have traditionally not done this. They may handle the temporary authorization requests for the PAN or use the Bank Identification Number (BIN) from the card number for routing the payments to the specific issuer, so they may need the number. That’s fine, as long as they store the data securely and have a log of who accessed the data and why the data was accessed.
Now that’s out of the way I can tell you what I’m doing, I’m playing with RSA Database Security Manager [now EOL’d] and RSA Key Manager. Simply put DBSM is a framework which encrypts the data as in goes into the database and decrypts it as it comes out. It’s something that anybody who is paranoid like me had already been doing for a while, but the way I was doing it required me to write custom fragments of code for every application which needed to access the data. DBSM does it transparently, while at the same time checking the users who try to access it, so only the correct users gain access. RKM hooks into this by providing a framework for the policies or controls which grants the correct people/devices/programs a key to lock-up or unlock the data, different policies can be implemented for different types of data or device.
Now you know what I do.
Originally appeared here.