Stargate: Towards DynamoDB Compatibility for Cassandra (Part II)

Stargate Architecture

How to Authenticate?

This is the first problem we encountered. InDynamoDB (and many other AWS products), there are multiple ways to authenticate, among which a common way is to set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environmental variables.

The Discrepancy in Data Types

Cassandra and DynamoDB have similar data models but they are not exactly the same. There is one-to-one mapping for basic types but not for the map, list, and set data types in DynamoDB. In DynamoDB, maps, lists, and sets can be deeply nested, meaning that you can have a list of maps of sets or even more complicated data structures. For example, you could insert an item whose goods attribute contains:

[
{
"name": "apple",
"price": 4,
"images": [
"apple1.jpg",
"apple2.jpg"
]
},
{
"name": "orange",
"price": 1.45,
"images": "orange.jpg"
}
]
goods list<frozen<map<text,text>>

Parser is All you Need

If you are building a database, then you almost certainly need to write parsers to parse the queries. DynamoDB, at the first glance, seemed to be an exception because it uses JSON as a request payload — there are so many JSON libraries that can help us with the parsing (deserialization). This seems to suggest we don’t need to worry about writing parsers. Unfortunately, this is not actually the case. DynamoDB queries have fields like FilterExpression that allow users to define certain conditions in plain text format. For example, in a query, you can have a filter expression like the following:

(Debt = :debt OR Deposit <> :deposit) AND Sex = :s

Conclusion

This concludes our journey toward the completion of our capstone project for our Spring semester in the Master of Computational Data Science (MCDS) program at Carnegie Mellon University (CMU). I would like to thank DataStax for the help and guidance throughout this journey. We will continue working on the project in the Fall semester to complete the rest of the APIs and do a thorough performance benchmark. Our hope is to deliver a complete product in the end!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Boxuan Li

Boxuan Li

Maintainer of JanusGraph, a popular distributed graph database