Visualizing world’s births and deaths

All data presented here is available as spreadsheet.

World population grows exponentially. Can we graph accurately world’s births and deaths since the dawn of the civilization? Of course not, but in this post I will try something (that I don’t think) anyone tried before. Idea is simple – every wikipedia article about a person has a category that is like “YYYY births” and “YYYY deaths” (if that person is dead). We can use this metadata to create a graph depicting births and deaths  in that specific year. I used Pywikibot to fetch this data. Here is the source code:

#!/usr/bin/env python
import wikipedia as pywikibot
import catlib
 
def main():
    site = pywikibot.getSite(code='en')
    for year in range(1, 2010):
        #cat = catlib.Category(site, "%s:%s" % (site.namespace(14), "%d_deaths" %year))
        cat = catlib.Category(site, "%s:%s" % (site.namespace(14), "%d_births" %year))
        print "%d - %d" % (year, len(cat.articlesList()))
 
if __name__ == '__main__':
    main()

And as already said, here is the spreadsheet with all the data. Before presenting results, just a small disclaimer – this approach has several disadvantages. First of all, results are collected from English Wikipedia, and as huge as it is, it is still western-centric, so probably in this statistics we are missing a lot of Chinese and other eastern world related persons. Secondly, a lot of people, especially those born in the distant past don’t have accurate year of birth or death. In a way, presented result will reflect our knowledge of the historical people’s life more than it will present world population growth, but some interesting results can be observed nonetheless. Thirdly, these are statistics just for notable persons, but I think, it can scale to whole population as well. Please feel free to comment whatever you see I missed or to further explain things on graphs where I don’t have explanations. All the data presented here is up to year 2009.

Births

This is graph of people’s births. On the x-axis are years, and on y-axis are number of births (click to enlarge):

births

Graph presented above is not very useful, so we’ll construct better one. Graph shown below is constructed by taking averages of births in range of 10 years (to see general trends) and is in logarithmic scale (due to the exponential nature of this kind of data). It reveals much more now (click to enlarge):

births-log

Now that we have these graphs, let’s see what we can conclude from them. We’ll start chronologically:

  • (red logarithmic graph) There is large rise of births around year 170, mostly all of them are Chinese military generals (you can see that here and here). I don’t know Chinese history, but they all relate to period of Three Kingdom era. Question now is – is this era period of great development in China or  is this the period when nothing important happened in Europe?
  • (red logarithmic graph) Around 600-1100 you can see there is no rise in birth’s graph. Of course, there should be, because population growth was constant, so I think this could be explained by Dark age period.
  • (red logarithmic graph) You can see rise in the first half of 17th century that I can’t explain. Is it large rise or large fall that happened after? Can that large fall be because of some catastrophic events like plague in Europe at that time? Or, there is large, but century long continuous rise because of also some large, but century long continuous catastrophe? Can baby boomer generation happen, but in large scale, spanning for decades? Or maybe it is just a pinnacle of renaissance, so there was a lot of famous people at that time?
  • (blue, linear graph) You can see very large rise in births after WWII (concrete year is 1947, you can see that in spreadsheet). I think this is very good visualization of baby boomer generation.
  • (blue, linear graph) Highest peak in births is around 1979-1985 (largest number of people born in Wikipedia is in 1982.), so I guess that you can, looking at this, see what is the most probable year of one’s affirmation and recognition. If you’re older than this, then your chances to become famous are becoming slimmer and slimmer every day:)

Deaths

Let’s take a look on number of deaths over the years. Here’s a linear graph (click to enlarge):

deaths

And here’s the same graph, but averaged in 10 years period and in logarithmic scale (click to enlarge):

deaths-log

Some observations I came up with:

  • (red logarithmic graph) The first thing you notice on this graph is a large peak around year 304. Reason is Diocletian persecution of Christians that happened that year and a large number of people that will later become Christian martyrs died. See for yourself.
  • (red logarithmic graph) Also interesting is a large drop of deaths around year 440 for which I really don’t have an explanation.
  • (red logarithmic graph) As in birth’s graph, here we also see a large rise in deaths between 16th and 17th century (or a large drop of deaths after that?) which  is also puzzling for me.
  • (blue, linear graph) Sudden jump around 1914-1918 is pretty clear. Deaths of millions, greatest tragedy ever, visualized as just a few pixels…
  • (blue, linear graph) Also jump around 1938-1945 shows how many great people died in WWII (and also millions of them unknown, that don’t have their place in Wikipedia…)
  • (red logarithmic graph) In contrast to birth’s graph that drops rapidly after 1985., graph of deaths just continues to grow, logically.

Is there anything else you can add to this observations that I missed? Can you explain better than me slopes in these graphs? Let’s squeeze all the facts we can from these data!

Posted in Analitika, Programiranje, Python | Tagged , , | 1 Comment

Allow access to whole S3 Bucket to IAM user

It took me a while to figure this out. Googling helped, but the answers are not obvious. So, you have IAM user and you want to grant that user complete read-write access to some bucket. Catch is that you need two statements to achieve this. Here is full bucket policy (just replace “YourIAMUser” and “YourBucketName” in the policy below):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
{
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Principal": {
                "AWS": "arn:aws:iam::821707826313:user/YourIAMUser"
            },
            "Resource": [
                "arn:aws:s3:::YourBucketName"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Principal": {
                "AWS": "arn:aws:iam::821707826313:user/YourIAMUser"
            },
            "Resource": [
                "arn:aws:s3:::YourBucketName/*"
            ]
        }
    ]
}

So, explanation now – as I already mentioned, notice that we have two separate statements (lines 3-14 and 15-28).

  • First one allow IAM user to “list buckets” (line 6) and resource given here is just plain ARN to the bucket (line 12)
  • Second statement gives that IAM user permissions on objects in bucket (lines 18-20), but resource given here is path to your bucket plus “/*” (line 26). This is the key thing I was missing when trying to create policy using AWS policy tool.

Hope this helps you!

Posted in Uncategorized | Leave a comment