![](\"img/logo-di.jpg\"){kind=logo}
\n",
"![](\"img/logo-demm.jpg\"){kind=logo}
\n",
"![](\"img/logo-unimi.jpg\"){kind=logo}
\n",
"\n",
"\n",
"\n",
"Organize data exploiting relations (that is, sets of tuples)
\n", "\n", "Among the most used:
\n", "\n", "![](\"img/table.png\")
\n",
"\n",
"Again, a wide variety:
\n", "\n", "This post is a good starting point in order to understand how they differ.
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Because the world is messy!
\n", "\n", "Incidentally, noSQL gives us interesting advantages:
\n", "\n", "MongoDB (http://www.mongodb.com) is a document-oriented open source noSQL DBMS.
\n", "\n", "MongoDB has, however, some critical points:
\n", "\n", "Simply put:
\n", "\n", "['one', 'two', 'three']\n", "\n", "
is much more simple than
\n", "\n", "```\n", "![](\"img/object.gif\")
\n",
"![](\"img/array.gif\")
\n",
"![](\"img/value.gif\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"| First Name | \n", "Last Name | \n", "License plate | \n", "
|---|---|---|
| Donald | \n", "Duck | \n", "313 | \n", "
| Mickey | \n", "Mouse | \n", "113 | \n", "
| Goofy | \n", "\n", " | 114 | \n", "
becomes
\n", "\n", "\n",
"{'first-name': 'Donald', 'last-name': 'Duck',\n",
" 'license-plate': 313},\n",
"{'first-name': 'Mickey', 'last-name': 'Mouse',\n",
" 'license-plate': 113},\n",
"{'first-name': 'Goofy', 'license-plate': 114}\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"MongoDB actually stores data using the BSON format (Binary JSON, http://bsonspec.org/), designed in order to:
\n", "\n", "Documents can be nested:
\n", "\n", "\n",
"{'first-name': 'Donald',\n",
" 'last-name': 'Duck',\n",
" 'car': {'model': 'American Bantam',\n",
" 'license-plate': 313}\n",
"}\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"Arrays are first-class objects in BSON:
\n", "\n", "\n", "\n",
"{'first-name': 'Donald',\n",
" 'last-name': 'Duck',\n",
" 'car': {'model': 'American Bantam',\n",
" 'license-plate': 313},\n",
" 'relatives': ['Huey', 'Dewey', 'Louie']\n",
"}\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"| `mongod` | server (listening on port 27017) |
| mongo | javascript client |
| API | for Python, Java, Scala, Ruby, C, C++, ... (http://api.mongodb.org) |
| installation | https://docs.mongodb.com/manual/installation |
\n",
"$ mongod --dbpath .\n",
"2018-05-28T08:47:04.813+0200 I CONTROL [initandlisten] MongoDB starting : pid=5172 port=27017 dbpath=. ...\n",
"...\n",
"\n",
"$ mongo\n",
"MongoDB shell version: 3.6.5\n",
"...\n",
">\n",
"\n",
"\n",
"Luckily, you shouldn't need to do that for the interactive part. Ready?
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Let's see who's who in Duckburg\n", "\n", "![](\"https://vignette.wikia.nocookie.net/scrooge-mcduck/images/1/11/Donald_Duck_Family_Tree.jpg/revision/latest/scale-to-width-down/1000?cb=20191124113106\")
\n",
"Source: Scrooge McDuck Wiki
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [], "source": [ "import pymongo\n", "client = pymongo.MongoClient('mongodb://mongodb:27017')\n", "duckburg_db = client.duckburg # silently creates db" ] }, { "cell_type": "code", "execution_count": 70, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/plain": [ "['admin', 'config', 'local']" ] }, "execution_count": 70, "metadata": {}, "output_type": "execute_result" } ], "source": [ "client.list_database_names() # it's empty, thus only default dbs will show up" ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 71, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ducks = duckburg_db.ducks # silently creates collection within db\n", "duckburg_db.list_collection_names() # it will not show up, either" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "The easiest way to put one document in a db is through the `insert_one` method" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [], "source": [ "r = ducks.insert_one({'first_name': 'Donald',\n", " 'last_name': 'Duck',\n", " 'gender': 'M',\n", " 'car': {'model': 'American Bantam',\n", " 'license_plate': 313},\n", " 'birth_year': 1920,\n", " 'first_appearance': 1934})" ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Based on the existence of specific fields
" ] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Donald Duck', 'Grandma Duck']" ] }, "execution_count": 86, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'car': {'$exists': True}})]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Selecting specific values
" ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Grandma Duck']" ] }, "execution_count": 87, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'gender': 'F'})]" ] }, { "cell_type": "code", "execution_count": 88, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Ludwig Von Drake']" ] }, "execution_count": 88, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'first_appearance': 1961})]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "On the basis of a relation
" ] }, { "cell_type": "code", "execution_count": 89, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Gyro Gearloose', 'Ludwig Von Drake']" ] }, "execution_count": 89, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'first_appearance': {'$gt': 1950}})]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "-" } }, "source": [ "Filtering more than one value for a field
" ] }, { "cell_type": "code", "execution_count": 90, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Scrooge McDuck', 'Ludwig Von Drake']" ] }, "execution_count": 90, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'first_appearance': {'$in': [1947,\n", " 1961]}})]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Logical conjunction
" ] }, { "cell_type": "code", "execution_count": 91, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Donald Duck', 'Scrooge McDuck']" ] }, "execution_count": 91, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'first_appearance': {'$lt': 1950}, \n", " 'gender': 'M'})]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Logical disjunction
" ] }, { "cell_type": "code", "execution_count": 92, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Donald Duck', 'Grandma Duck']" ] }, "execution_count": 92, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'$or': [{'birth_year': {'$gt': 1900}}, \n", " {'gender': 'F'}]})]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Array contents
" ] }, { "cell_type": "code", "execution_count": 93, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Gyro Gearloose', 'Ludwig Von Drake']" ] }, "execution_count": 93, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'hobbies': 'study'})]" ] }, { "cell_type": "code", "execution_count": 94, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Ludwig Von Drake']" ] }, "execution_count": 94, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'hobbies': ['study']})]" ] }, { "cell_type": "code", "execution_count": 95, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Gyro Gearloose']" ] }, "execution_count": 95, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'hobbies.1': 'study'})]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Embedded documents
" ] }, { "cell_type": "code", "execution_count": 96, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Donald Duck']" ] }, "execution_count": 96, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[show_duck(d) for d in ducks.find({'car.model': 'American Bantam'})]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Regular expressions
" ] }, { "cell_type": "code", "execution_count": 97, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Donald Duck', 'Grandma Duck', 'Scrooge McDuck']" ] }, "execution_count": 97, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import re\n", "regx = re.compile(\"uck$\", re.IGNORECASE)\n", "\n", "[show_duck(d) for d in ducks.find({'last_name': regx})]" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "-" } }, "source": [ "\n", "\n", "Using regular expressions, as well as `$lt` or `$gt`, may lead to inefficiency in query execution." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Implemented via a second argument to `find`\n", "\n", "Collections can be indexed specifying a set of fields and the corresponding sort order (the syntax used by pymongo is in this case different from the original API)
" ] }, { "cell_type": "code", "execution_count": 112, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'first_name_1_first_appearance_-1'" ] }, "execution_count": 112, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ducks.create_index([('first_name', pymongo.ASCENDING),\n", " ('first_appearance', pymongo.DESCENDING)])" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "If a query is covered by an index, that is\n", "\n", "
are within the index, the query is executed via index traversal.
" ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{'first_name': 'Grandma', 'first_appearance': 1943},\n", " {'first_name': 'Gyro', 'first_appearance': 1952}]" ] }, "execution_count": 113, "metadata": {}, "output_type": "execute_result" } ], "source": [ "regx = re.compile(\"^G.*\", re.IGNORECASE)\n", "list(ducks.find({'first_name': regx, 'first_appearance': {'$lt': 1955}},\n", " {'_id': 0, 'first_name': 1, 'first_appearance': 1}))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "`find` returns a cursor to the query resultset, accessible via\n", "\n", "- invocation of the `next` method (until `StopIteration`)" ] }, { "cell_type": "code", "execution_count": 114, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Donald Duck\n", "Grandma Duck\n", "Scrooge McDuck\n", "Gyro Gearloose\n", "Ludwig Von Drake\n" ] } ], "source": [ "cursor = ducks.find()\n", "try:\n", " while True:\n", " print(show_duck(cursor.next()))\n", "except StopIteration:\n", " pass" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "-" } }, "source": [ "- positional access through `__getitem__` (aka `[]`)" ] }, { "cell_type": "code", "execution_count": 115, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Grandma Duck'" ] }, "execution_count": 115, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cursor = ducks.find()\n", "show_duck(cursor[1])" ] }, { "cell_type": "code", "execution_count": 116, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Ludwig Von Drake'" ] }, "execution_count": 116, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_duck(cursor[4])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(even going back)" ] }, { "cell_type": "code", "execution_count": 117, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Scrooge McDuck'" ] }, "execution_count": 117, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_duck(cursor[2])" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "-" } }, "source": [ "- `list` conversion, `for`-based iteration," ] }, { "cell_type": "code", "execution_count": 118, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Donald Duck\n", "Grandma Duck\n", "Scrooge McDuck\n", "Gyro Gearloose\n", "Ludwig Von Drake\n" ] } ], "source": [ "cursor = ducks.find()\n", "\n", "for d in cursor:\n", " print(show_duck(d))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "-" } }, "source": [ "- invocation of the `skip`, `limit`, `sort`, and `count` methods" ] }, { "cell_type": "code", "execution_count": 108, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Ludwig Von Drake\n", "Scrooge McDuck\n", "Gyro Gearloose\n", "Donald Duck\n", "Grandma Duck\n" ] } ], "source": [ "cursor = ducks.find().sort('last_name', pymongo.DESCENDING)\n", "for d in cursor:\n", " print(show_duck(d))" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "This (deprecated) form of update completely replaced the document!
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Use the `update_one` and `update_many` methods" ] }, { "cell_type": "code", "execution_count": 122, "metadata": { "scrolled": false }, "outputs": [ { "ename": "ValueError", "evalue": "update only works with $ operators", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m[{'name': 'invention', 'day': 'Monday'},\n",
" {'name': 'study', 'day': 'Friday'}]\n",
"\n",
"The update query modifies the day for the study hobby:
\n", "\n", "{{'hobbies.name': 'study'},\n",
" {'$set': {'hobbies.$.day': 'Tuesday'}}}"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"What happens if one attempts to update a non-existing document?
\n", "\n", "- nothing...\n", "- ...unless the `upsert=True` additional argument has been specified in `update_one` or `update_many`: in this case the operation results in an insert-upon-update (upsert)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "...at the price of an additional query (to recover, in this case, the uncle).
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "Many-to-many joins can be implemented
\n", "\n", "\n",
"ducks.insert_one({\n",
" 'first_name': 'Donald',\n",
" 'last_name': 'Duck',\n",
" 'relatives': [ObjectId('516405b8b356cd6125b74e89'),\n",
" ObjectId('516405b8b356cd6125b74e8a'),\n",
" ObjectId('516405b8b356cd6125b74e8b')]})\n",
"ducks.find({\n",
" 'relatives': ObjectId('516405b8b356cd6125b74e8a')})\n",
"\n",
"\n",
"\n",
"ducks.insert_one({\n",
" 'first_name': 'Donald',\n",
" 'last_name': 'Duck',\n",
" 'relatives': [{'first_name': 'Huey'},\n",
" {'first_name': 'Dewey'},\n",
" {'first_name': 'Louie'}]})\n",
"ducks.find_one({'relatives.first_name': 'Dewey'})\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"\n",
"resource date\n",
"index Jan 20 2010 4:30\n",
"index Jan 20 2010 5:30\n",
"about Jan 20 2010 6:00\n",
"index Jan 20 2010 7:00\n",
"about Jan 21 2010 8:00\n",
"about Jan 21 2010 8:30\n",
"index Jan 21 2010 8:30\n",
"about Jan 21 2010 9:00\n",
"index Jan 21 2010 9:30\n",
"index Jan 22 2010 5:00\n",
"\n",
"\n",
"Starting from these data...\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"...we want to obain\n",
"\n",
"\n",
"resource year month day count\n",
"index 2010 1 20 3\n",
"about 2010 1 20 1\n",
"about 2010 1 21 3\n",
"index 2010 1 21 2\n",
"index 2010 1 22 1\n",
"\n",
"\n",
"We can use a MapReduce job in which\n",
"\n",
"- map emits 1 for each (resource, date) key\n",
"- reduce sums up values"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"We populate the database at first"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {},
"outputs": [],
"source": [
"import datetime\n",
"\n",
"ducks.hits.insert_one({'resource': 'index',\n",
" 'date': datetime.datetime(2010, 1, 20, 4, 30, 0, 0)})\n",
"ducks.hits.insert_one({'resource': 'index',\n",
" 'date': datetime.datetime(2010, 1, 20, 5, 30, 0, 0)})\n",
"ducks.hits.insert_one({'resource': 'about',\n",
" 'date': datetime.datetime(2010, 1, 20, 6, 0, 0, 0)})\n",
"ducks.hits.insert_one({'resource': 'index',\n",
" 'date': datetime.datetime(2010, 1, 20, 7, 0, 0, 0)})\n",
"ducks.hits.insert_one({'resource': 'about',\n",
" 'date': datetime.datetime(2010, 1, 21, 8, 0, 0, 0)})\n",
"ducks.hits.insert_one({'resource': 'index',\n",
" 'date': datetime.datetime(2010, 1, 21, 8, 30, 0, 0)})\n",
"ducks.hits.insert_one({'resource': 'about',\n",
" 'date': datetime.datetime(2010, 1, 21, 9, 0, 0, 0)})\n",
"ducks.hits.insert_one({'resource': 'index',\n",
" 'date': datetime.datetime(2010, 1, 21, 9, 30, 0, 0)})\n",
"_ = ducks.hits.insert_one({'resource': 'index',\n",
" 'date': datetime.datetime(2010, 1, 22, 5, 0, 0, 0)})"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"We subsequently write the map function (in javascript!)"
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {},
"outputs": [],
"source": [
"from bson.code import Code\n",
"\n",
"map = Code('''function() {\n",
" var key = {\n",
" resource: this.resource,\n",
" year: this.date.getFullYear(),\n",
" month: this.date.getMonth(),\n",
" day: this.date.getDate()\n",
" };\n",
" emit(key, {count: 1});\n",
"}''')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"And we create analogously the reduce function"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {},
"outputs": [],
"source": [
"reduce = Code('''function(key, values) {\n",
" var sum = 0;\n",
" values.forEach(function(value) {\n",
" sum += value['count'];\n",
" });\n",
" return {count: sum};\n",
"}''')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"_et voilà!_"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Collection(Database(MongoClient(host=['mongodb:27017'], document_class=dict, tz_aware=False, connect=True), 'duckburg'), 'hit_stats')"
]
},
"execution_count": 64,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"result = ducks.hits.map_reduce(map, reduce, 'hit_stats')\n",
"result"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'_id': {'resource': 'about', 'year': 2010.0, 'month': 0.0, 'day': 20.0}, 'value': {'count': 1.0}}\n",
"{'_id': {'resource': 'about', 'year': 2010.0, 'month': 0.0, 'day': 21.0}, 'value': {'count': 2.0}}\n",
"{'_id': {'resource': 'index', 'year': 2010.0, 'month': 0.0, 'day': 20.0}, 'value': {'count': 3.0}}\n",
"{'_id': {'resource': 'index', 'year': 2010.0, 'month': 0.0, 'day': 21.0}, 'value': {'count': 2.0}}\n",
"{'_id': {'resource': 'index', 'year': 2010.0, 'month': 0.0, 'day': 22.0}, 'value': {'count': 1.0}}\n"
]
}
],
"source": [
"for doc in result.find():\n",
" print(doc)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"MongoDB also provides weaker forms of aggregation which do not require the use of map-reduce algorithms.\n",
"\n",
"This is done using the `aggregate` method in conjunction with operators such as `$match`, `$project`, `$group`, `$sort`, etc."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"\n",
"There's no universal solution: the choice depends on the problem under study, as well as on personal design style
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "\n", "\n", "\n",
"mongoimport --db books --collection books --file catalog.books.json\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"Although we saw that a tabular description might not be the best resource to be used when visualizing noSQL data, we can build and show a Pandas dataframe as a quick way to peek into the dataset."
]
},
{
"cell_type": "code",
"execution_count": 113,
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | _id | \n", "authors | \n", "categories | \n", "isbn | \n", "longDescription | \n", "pageCount | \n", "publishedDate | \n", "shortDescription | \n", "status | \n", "thumbnailUrl | \n", "title | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "3 | \n", "[Gojko Adzic] | \n", "[Software Engineering] | \n", "1617290084 | \n", "NaN | \n", "0 | \n", "2011-06-03 | \n", "NaN | \n", "PUBLISH | \n", "https://s3.amazonaws.com/AKIAJC5RLADLUMVRPFDQ.... | \n", "Specification by Example | \n", "
| 1 | \n", "1 | \n", "[W. Frank Ableson, Charlie Collins, Robi Sen] | \n", "[Open Source, Mobile] | \n", "1933988673 | \n", "Android is an open source mobile phone platfor... | \n", "416 | \n", "2009-04-01 | \n", "Unlocking Android: A Developer's Guide provide... | \n", "PUBLISH | \n", "https://s3.amazonaws.com/AKIAJC5RLADLUMVRPFDQ.... | \n", "Unlocking Android | \n", "
| 2 | \n", "2 | \n", "[W. Frank Ableson, Robi Sen] | \n", "[Java] | \n", "1935182722 | \n", "When it comes to mobile apps, Android can do a... | \n", "592 | \n", "2011-01-14 | \n", "Android in Action, Second Edition is a compreh... | \n", "PUBLISH | \n", "https://s3.amazonaws.com/AKIAJC5RLADLUMVRPFDQ.... | \n", "Android in Action, Second Edition | \n", "
| 3 | \n", "6 | \n", "[Satnam Alag] | \n", "[Internet] | \n", "1933988312 | \n", "There's a great deal of wisdom in a crowd, but... | \n", "425 | \n", "2008-10-01 | \n", "NaN | \n", "PUBLISH | \n", "https://s3.amazonaws.com/AKIAJC5RLADLUMVRPFDQ.... | \n", "Collective Intelligence in Action | \n", "
| 4 | \n", "4 | \n", "[Tariq Ahmed with Jon Hirschi, Faisal Abid] | \n", "[Internet] | \n", "1933988746 | \n", "New web applications require engaging user-fri... | \n", "576 | \n", "2009-02-02 | \n", "NaN | \n", "PUBLISH | \n", "https://s3.amazonaws.com/AKIAJC5RLADLUMVRPFDQ.... | \n", "Flex 3 in Action | \n", "