SmexyyWeby

Scripting for Fun and Passion

Statistical Analysis of Rahul versus Arnab


So, Arnab kept asking about Rahul’s opinion on Modi, 1984 riots and Ashok Chavan and Rahul fought him back with RTI, women empowerment and broader system related questions from his armory. This is how one of India’s recent probably “Once in a Lifetime” faceoff between 2 social media hot favorites ended. The unstoppable force versus the immovable object. I present a data based analytics of the whole proceeding.

It has been almost a week since Rahul Gandhi’s interview with Times Now journalist Arnab Goswami was published on Youtube. For those of you who haven’t seen the recent hot thing in Indian politics should spend some 1.5 hours of your time studying the psych of the Vice President of our current party. 

Since its publishing the video has garnered more than 1.7 million views and its has has been quite a viral thing in the days following the actual interview.

Youtube Video Stats. Source: https://www.youtube.com/watch?v=xB_eWW5ttaM

Youtube Video Stats. Source: https://www.youtube.com/watch?v=xB_eWW5ttaM

This has also allowed the politically engrossed Indian masses on social media to share their sentiments about Rahul Gandhi and his interview. The comments in my friend groups have been mostly funny and quite humorous. A majority of them have claimed that the interview was all about Rahul reiterating the same points over and over again. Things like “Women Empowerment”, “RTI” and “Rahul Gandhi” were supposedly some of the words which were supposed to be overused by Rahul. The social media was abuzz with memes about Rahul Gandhi [Source: http://www.india.com/whatever/rahul-gandhis-interview-with-arnab-goswami-the-best-tweets-and-jokes-9234/] and there was even a website totally dedicated to generating answers as Rahul would have given. [Source: http://engagedino.com/askrg]

Being a data scientist and a starter in text processing I decided to do a fun weekend project on the interview text and look for patterns and if they are correctly correlated to the claims people are making on social media. Another reason this interview was of particular interest to me because it bought 2 icons of Indian media together. It was like “an unstoppable force meeting an immovable object” [Source: The Dark Knight, 2008] and I am sure the people saw scales remaining balanced till the end.

I looked at the data from 3 perspectives:

  • All Data
  • Rahul’s Text
  • Arnab’s Text

This was important because I wanted to do a frequent statistical analysis and try to see if the claims on social media were correct. So I decide to answer the following research question:

“How accurate are the claims on social media about Rahul versus Arnab and what insights do they give into the personalities of the 2 involved entities ?”

With this simple question in mind I decided to first test word frequencies of all the 3 datasets and some of the preliminary results were not quite consistent with the claims and reflected the social media audience’s inclination to hang on to some catch phrases from the interview and make a whole viral campaign out of it.

Looking at all the data cumulatively the hot topics which were quite prominent during the interview were: riots, system, RTI and Gujrat. Now this is quite understandable as Arnab was trying to focus on issues like Gujrat riots and Rahul was focused on the things related to system changes as a part of his broader perspective strategy.

A more statistical result was:

All word statistics

All word statistics

However, a more interesting thing I was interested was in the number of entities mentioned. This allowed me to focus on key individuals who were mentioned during the interview. And the results I got were quite interesting. Leaving out Rahul [Rahul will be discussed in detail when studying Rahul's text independently ;)] and Congress, the other key entities were Gujrat and Narendra Modi which is also quite evident. However, the most interesting entity which surfaced was Ashok Chavan whose name Arnab used a lot of times to extract some answers from Rahul. Also 1984 and Cambridge were entities discussed quite frequently.

All Text Entities

All Text Entities

I also constructed a network of entities which occurred together and these results reflected similar patterns. Also some people whose names were linked to the 1984 riots like Sajjan Kumar, Bhagat, Jagdish Tytler etc. were also evident from the analysis.

All Entity Network

All Entity Network

To try to find out the central theme of the interview I did topic modelling of the text and got 5 major cluster of topics which co-occurred frequently. Apart from central theme being Rahul v/s Modi and the elections of 2014, the other important but more frequent themes were hidden mostly in Rahul’s answers regarding women issues, RTI, system and the 2 riots of Gujrat and 1984. Ashok Chavan was also frequently used during interview regarding him being shielded in the Adarsh Scam.

Frequent and important topics during interview

Frequent and important topics during interview

Once finished with the overview analysis of the text corpus as a whole I decided to dig deeper and look into the individual statements given by both Rahul and Arnab. This is the interesting dataset according to me as this will give me answers to the research question I was pursuing.

On looking at Arnab’s dataset it was quite evident that he continued his style of asking very detailed questions Arnab spoke around 5071 words as compared to Rahul’s 7460. While Arnab was focused on issues like Modi, Chavan and 1984 riots; Rahul was more focused on issues like system, people, RTI and women. However, the internet memes started to get visualized when I looked at the entity results of Rahul and Arnab. While Arnab mentioned entities like Rahul, Modi, Gujrat and 1984; Rahul’s top entities included Congress, Gujrat, India and “Rahul Gandhi” [The Rock and Stone Cold Steve Austin would be amazed at the new entry to their club].  Infact Rahul used Rahul Gandhi 11 times during his statements, more than the number of times he used Modi [6] or even Ashok Chavan[3].

Rahul's Word Stats

Rahul’s Word Stats

Rahul's Entity Stats

Rahul’s Entity Stats

Arnab's Word Frequency

Arnab’s Word Frequency

Arnab's Entity Stats

Arnab’s Entity Stats

Another interesting thing I found was that BJP and AAP were very less frequently used by both individuals, especially when compared to the the number of times Modi and Congress were mentioned.

Entity

Rahul

Arnab

Modi

6

56

Congress

59

33

BJP

3

9

AAP

1

8

Rahul Gandhi

11

90

Ashok Chavan

3

26

While Arnab’s questions revolved around topics related to Modi, Congress, Chavan and Riots; Rahul’s answers were mostly about RTI, System, youngsters in election with the central topic revolving around women issues. The central topics were not the most frequent ones but the ones which were most uniformly distributed in the whole conversation.

Arnab's Entity Network

Arnab’s Entity Network

Rahul's Entity Network

Rahul’s Entity Network

Rahul in his statements tried to connect Congress party to issues related to RTI, India along with focusing on its performance in various states. Rahul frequently tried to draw differences between Gujrat riots and 1984 riots. This was quite different from the entities Arnab tried to link. Arnab’s focus revolved around Modi and his comments of Shehzada about Rahul, Rahul’s performance in UP. Arnab also tried to pit Rahul against the BJP PM candidate Modi by bringing Modi’s candidature for the PM of India, quite regularly during the interview.

Arnab's Topics

Arnab’s Topics

Rahul's Topics

Rahul’s Topics

Method

I am thankful to the Times of India website for making the whole Interview script text publicly available online. [Source: http://timesofindia.indiatimes.com/india/Rahul-Gandhis-first-interview-Full-text/articleshow/29455665.cms%5D. This made the text analysis a far more easy project to me. (After all I was not interested in spending another hour and a half trying to transcribe the whole audio).

After getting the data I had to clean it to get it into analytically state. I decided to split it into 3 separate data-sets:

  • Full text of Interview
  • Only Rahul’s Text
  • Only Arnab’s Text

I used the tool called ConText for data analysis like word stats, entity stats, network generation and topic visualizations along with some python scripts for parsing the data.  And I created the visualizations in Gephi using centrality measures for Node sizes [Degree] and Label Sizes [Betweenness] and modularity classes for node coloring. 

Interactive Charts of the images presented above along with full analytics data can be found at:

https://docs.google.com/spreadsheet/pub?key=0AipE5c521mI_dENtQWdncUNsRk56Y0l5LWxxczBTakE&gid=0

Conclusions

After doing this basic analysis I realize I figured out that even though the topics which were not frequent but were uniform in the discussion they became more popular in social media. Rahul’s usage of women empowerment, RTI and “Rahul Gandhi” were caught by social media enthusiasts and made viral. However, this also led to many other important topics and issues being hidden beneath this viral sharing. Key individuals like Ashok Chavan, Virbhadra and some scams which were mentioned were not caught by the social media audiences.

Another important observation was regarding evading of questions by Rahul and how less he tried to answer to the point or pointing out the individuals who he was supposed to give statement on. Even though it is a perfectly safe and good strategy to answer in a positive tone mentioning issues which one envisions; I would say that when it comes to personal interview being a bit more specific and elaborate on the questions at hand is more important. As the statistics reflected the platform appeared more to me as a means to talk more about what he is planning for the future and has done in the past than  about what are the key things at in the current political scenario. Overall the claims on social media were quite accurate.

Finally, I still feel there are lot more things which can make this analysis more useful and interesting. Some ideas I have but can’t implement because of lack of time [PhD studies ;)] are:

  • Word correlation on for each question and its corresponding answer
  • Language model for Rahul Gandhi’s answers and Arnab’s question [the latter can be done more easily because of the abundant dataset available]
  • Sentiment tracking for each entity and in what way the answer’s were given.

Fun Bites

Today only I also happened to see this quite dramatic reconstruction of the whole interview by Cyrus Broacha. I think the language model for both Rahul and Arnab would have greatly improved the video.

Legals

This article is my personal analysis and opinion on the issue at hand. I have cited sources from which I have taken the data and the tools I have used. If anyone plans to reproduce this article or my analysis on their site, please give a link back.

If you agree or disagree, or have thoughts to add to my analysis or want to answer more broader questions to my analysis related to light bulb changing labor forces and chickens crossing the streets. Please feel free to use the comment section.

Also, humor and analysis were some of the key elements I thought of while writing this piece and I am pretty sure I ended up doing the later relatively more than the former.

2013 – An Unexpected Journey


Welcome to 2014

Welcome to 2014

Live Life Like A Jive – Yes, those were the words which defined and shaped my year 2013. This was an year which included a lot of learning experiences for me. As I sit here to reminiscence the 365 days which just flew past me like a gush of fresh air, I remember the life they breathed into me as they passed by. I want to talk about a few key things which 2013 added to my life. It is going to be long but compressing such an eventful year to a byte would be very wrong on my part of celebration.

Moments

Time for me was never about the amount of hours or days spent involved in something or the other. But it was always about moments, moments which I can cherish, moments which I can learn from, moments which shape me and moments which challenge me. 2013 was a lot about creating those moments, a majority of them with people I learned to like and cherish in my life. Some of the key moments helped me imbibe the feeling of happiness, attachment, exuberance, humility and creativity.

Some moments were spent alone exploring my own self, I did this by indulging in my new found love for creative writing especially poetry. I penned a few poems in Hindi and English continuing with my efforts from last year. A few I recited to some close friends another just kept in my diary. Some I never even bothered to write them down, instead I made them into my own lullaby.

Some moments were of learning a quintessential amount of what the world has to offer me, this again was a continuation of what 2012 has springboarded me into. I continued with my efforts of learning on Coursera and finished my almost 1 academic year in coursera with 6 finished course with certificates and 1 finished but un-certified experience. I realized that the certificates were just a way to keep me focussed in trying to take the course seriously. They were not the end goal but an important force in making me learn what I was giving time to. I have been very satisfied with my indulgence in online learning platforms and want to continue it for the rest of my life.

A few moments were of exploring the world around me, not just the natural but also the man made one. I would attribute this to some amazing journeys I commited myself to undertake and was fortunate to be made a part of. Partaking the adventurous and enduring trek experience at Mullayangiri ranges was an experience to remember. Where I lived what I would have wanted in my dreams sometimes, to wander like Frodo in the Mountains and reach the end at the end of the day. It was a personal achievment for me finishing that 15 km trek and realizing that life has so much more in store for me if only I continue taking one step at a time in the forward direction. I was fortunate to top it up with some really amazing trips with my new fellowship of friends I found in Bangalore, a place I would always remember that 1 year in Bangalore as a turning point of my life till now.

Some moments of composing my jives with my old friends, to eating dinner and watching F.R.I.E.N.D.S. with my old college buddies. Going on bike rides in rainy days and all the random restaurant trips during late night hours. To my new found road side aloo paratha dabhas and some road trips in car with no random plans.

The moments of performing our choreographed sequence in front of a housefull crowd and working with one of the best teams I could have got in my first workplace out of college.

The joyous moment of finally being able to pursue my boyhood dream of doing research and along with the sad realization of leaving all that I learned and earned in an year to move forward to pursue bigger goals in my life. The realization of the fact that everything that has a beginning has an end. But I was happy that this beautiful joruney of my life ended on a high note.

The moments spend in theaters watching some of the finest bollywood and hollywood movies of the year with people I love discussing them with.

The moment of watching my first cricket match in the stadium and watching one of my inspirations Adam Gilchrist and my crush Priety Zinta at their finest.

The new moments were of spending probably one of the longest in years to come and best time with my family before I set forth to the new world.

The uncountable moments of photoboothing in various UIUC events, and countless magic moments with the new people and reuniting with some close old people in my life.

On an academic level the moments were of seeing our first reseach output being published to demonstrating our tool to a whole breed of social change agents. The slight sadness at missing the perfect 4/4 because of an A- but at the sametime what realizing what research is all about. The constant efforts towards writing my first paper which are still underway.

The proud feeling of once again starting on an idea, building and working with great a team and successfully getting accepted for our efforts. More on this one after next year March ;)

Being awed by the beauty of life like a couple kissing in their car parked at the top most turning slope of the Lombard street, SFO, the journey atop the Blackwell forest reserve, the glad you came trip and the dangerous WOLF tour. The first concert experience and the countless free stuff ;)

The many delicious meals I learned to cook and serve, and the innumerable dine-ins and eat-outs with my new friends. The late nighters and the chai latte, maggie after that. The 1 am effort to cook pudding and gifting it to my Professor =)

The long walks and bus rides, the sleep-ins and nightouts.

The feeling of being hugged by a thousand kids, dressed and dancing around like Martha – the speaking dog, while sweating like a pig inside the mask.

The snow angels and the youtube guided snowman. The eyebrow raising at the Halloween Frat party and my DIY costume.

The so much to do and yet so less time in hand. 2013 was like an alfresco ride in an Amazon forest.

A lot of moments with myself but many of those because of the people who I am glad to have shared those moments with.

People

There were a lot of people who made this year so fantastic for me. Right from my family’s and some people’s encouragement in helping me decide my future to my team, my manager, my advisors and my friend’s shaping every single day of my year. Some people with whom new bonds were forged and some with which old ties were strengthened. The people who were part of my journeys, my adventures and my wisdom. The people who were my guest and the ones who were my host. The new cohorts and the closer team mates. The old friends I left back in India and the new ones I made in US. The so many birthday boys and girls I had a chance to celebrate with.

And yes my own self who absorbed all this and continued on the the smile =)

Events

All the moments and all the people were part of few major events in my life.

- My last few months at Citrix, which involved a long session of lunches, treats, trips, birthday celebrations and farewells.
– The time spend at my home in Bangalore and with friends involving a majority of the moments I remember.
– The long academic and social six month period at UIUC and USA in general.

What Next –>

The year is gone and the story is written. As the clock ticks on the new year and new day is passing by. I have new moments to create, new people to meet, new places to go, new things to learn and lot more things to DO but most importantly to continue on with my motto “Live Life Like A Jive” and as Frost said “miles to go before I sleep …”.

Happy New Year for another unexpected Journey =)

PS: I am actually going to sleep right now as it is too late and I am feeling sleepy =P

Jeevan ki vo baaten – जीवन कि वो बातें


क्या पता है तुमको,
कि दुनिया में सबसे ज्यादा,
दर्द क्या देता है ।

वो अनकही सी बातें,
जो इंसान दिल में दबा रखता है ।
अगर केह दिया होता उन लवजून को,
तो शायद ये डगर सुहानी हो जाती ।
पर न कहा हमने,
न जाने क्यूँ ।

शायद ये गुरूर था,
या फिर था डर ।
कुछ खोने का,
या खुद झुकने का ।

उन शब्दों को दबा के रखा है,
जो संजोये थे किसी और के लिए ।
कुछ उस दिलबर के लिये,
तो कुछ उस दोस्त के लिये,
कुछ थे अपने लाडले के लिये,
और कुछ थे परिवार के लिए ।

क्यों ना आज,
वो बातें हम बोल दें ।
अपने उन रिश्तों में ,
फिर से नया रस घोल दें।
वो क्षमा, वो इकरार,
वो दोस्ती का इज़हार,
वो ममता, वो प्यार ।

क्यूँ ना आज सब कुछ,
हाँ सब कुछ उड़ेल दे ।

और भर ले इस दिल को,
कुछ खुशनुमा लम्हों से ।
हाँ जी ले ये ज़िन्दगी,
इन नयी उमंगों में ।

Haha – Vo Parchhaiyan: हाहा – वो परछाईयां


Image

परछाइयों का पीछा करते करते,
मैं भूल गया था के वो लोग कौन थे,
भूल गया की कब धुप थी ,
और कब थी छान्व।

भागता रहा मैं बस अपनी मर्ज़ी से,
पता नहीं किस डगर पर,
जब पैर थामे तो ये था पाया ,
की जो मैं चाहता था पकड़ना ,
वो था मेरा अपना साया।

आज देखता हूँ बीते उस पल को,
तो भीनी सी मुस्कान है चेहरे पे आति,
की उस दौड़ में भी एक मस्ती थी,
एक आराम देने लायक थकान थी।

आगे कुछ और है रस्ते,
और दूर कुछ और हैं परछाईयां ,
पर इस बार थोडा दूर चल लूं ,
कुछ आहिस्ता लम्हे गुज़ार लॊन,
और हसीं इस सफ़र को,
मैं थोडा और संवार लूं ।

On Modi to non-Indians


This is a note I wrote to describe a bit of details about Narendra Modi, the current CM of Gujrat and the PM candidate by BJP for 2014 National elections of India. The note was written as a reply to some of my foreign friends who have heard about the 2002 riots and have a opinion that he has committed a very high human rights violation by being responsible for the 2002 riots. This came after I posted a few updates in which there might have been a mention of Narendra Modi [http://www.facebook.com/shubhanshu.mishra/posts/10151610104401434]. I believe they have a point which is valid considering the fact that the only popular news about Modi is the involvement in 2002 riots. But I was a bit surprised to see its impact on people on people outside India. So, I decided to just put forth my opinion but back it up with some news fact, website which I have to believe as true. Below is my reply:

Narendra Modi

Narendra Modi

I have done quite a bit of study on the 2002 riots and the Indian supreme court after continuous allegations on him have declared him not guilty. He was not responsible for the riot in the first place as the riot started because of a very grave reason and a long history. The riots were a part of a linked set of events since the Bombay riots. He himself told the press that if you can prove me guilty you can hang me. And right now his development model and his way of thinking about progress is a new hope for India. There have been many riots in India where bad things have happened [http://en.wikipedia.org/wiki/1984_anti-Sikh_riotshttp://en.wikipedia.org/wiki/List_of_massacres_in_India] but we don’t see those riots being publicized too much because they were by the ruling government.

But that apart. He is the one of the few persons who talks about bringing India to the top of the world. He talks of not judging individuals on basis of casts, religion but of working towards progress of Nation. [http://articles.economictimes.indiatimes.com/2013-09-16/news/42114299_1_narendra-modi-prime-ministerial-candidate-gujarat-chief-minister]

There is a huge difference between being involved and failure to control. His was the latter case, even though as CM and as any leader one has to bear the brunt of any bad thing which happens in their regime. However when the riot happened he was just 3 months into his term and even after the riot the people of Gujrat have been electing him with huge majority for past 3 years which includes the immediate election in 2002. [http://en.wikipedia.org/wiki/Gujarat_legislative_assembly_election,_2002] And his development practices [http://www.narendramodi.in/a-journey-through-gujarat%E2%80%99s-development-through-power-point-presentations/][http://www.indianexpress.com/news/what-is-the-gujarat-model-/1045851/0] have become world famous and people really consider the growth of Gujrat state both economically and human resources wise as a best case practice. The state is known for having a very efficient and electricity surplus production[http://ibnlive.in.com/news/gujarat-24hour-power-supply-in-over-18000-villages/250309-3.html] as compared to other states of India. And this changed after 2001 before which it was a state reeling with electricity crisis. [http://businesstoday.intoday.in/story/gujarats-power-sector-turnaround-story/1/21750.html]

A major reason why we all are supporting him is that in a country where riots and communal discrimination is widespread he is an individual who talks of solving basic problems of success, resources and instilling a self confidence in the country. I would always want a person to be a PM in whose CM term the women feel safe[http://news.oneindia.in/feature/2012/gujarat-championing-cause-of-women-empowerment-1022167.html], the kids are properly fed[http://articles.economictimes.indiatimes.com/2013-02-20/news/37200103_1_plan-outlay-child-healthcare-budgetary-allocation] and educated and the youth is given employment.

In fact even the majority of Muslim people of Gujrat praise his development models. [6 out of 8 seats of Muslim majority seats were won by his party, http://ibnlive.in.com/group-blog/the-india-blog/3581/why-gujarati-muslims-are-with-modi/64627.html ]

To quote a few Recently, India’s most well-known film script-writer Salim Khan (actor Salmaan Khan’s father) has said to a senior journalist in an interview:

“Does anyone remember who the chief minister of Maharashtra was during the Mumbai riots which were no less deadly than the Gujarat riots of 2002? Does anyone recall the name of the chief minister of UP during Malliana and Meerut riots or that of the Bihar CM when the Bhagalpur or Jamshedpur riots under Congress regimes took place? Do we hear names of earlier chief ministers of Gujarat under whose charge, hundreds of riots took place in post-Independence India? Does anyone remember who was in-charge of Delhi’s security when the 1984 massacre of Sikhs took place in the capital of India? How come Narendra Modi has been singled out as the Devil Incarnate as if he personally carried out all the killings during the riots of 2002?”

[Read more at: http://ibnlive.in.com/group-blog/the-india-blog/3581/why-gujarati-muslims-are-with-modi/64627.html?utm_source=ref_article]

So overall, I choose not to be affected too much by the propaganda and choose to believe the facts out there and results I can see, every one is good or bad and every one is accused and praised by people/mob sometimes. What I liked was the fact that Modi choose not to run away or cover up things using dirty politics but by driving growth and progress. A lesson I feel every leader should learn. Even King Ashoka was a big destroyer but after the Kalinga war he shifted to Buddhism and choose to walk the path of peace. Now people praise him even though the war of Kalinga was utterly destructive. The reason people praise him is for what he DID. [http://en.wikipedia.org/wiki/Ashoka]

I just wanted to share this with you so that you can look at this topic with open mind. Yes the riot was very bad thing and as the then current CM he has to take responsibility for everything that happened, which he did by taking enough steps in trying to stop it, but that is it. If a person is doing good now, I choose to forget the past which comes with him/her. The whole point of what we all preach in this world today is that everyone can change in order to become a better person but what is the point of all that preaching if we don’t accept the changed person and keep saying that he did something bad once and his change is insignificant in that light.

I hope this helps you with the topic, plus I usually don’t write praising or criticizing him, its just that if there is news about the country and I want to share I do that. I am sure you will be able to ignore these news or might read them peacefully in future =)

He is not my hero or anything, just a political figure of India. And I choose to be rational about him using the facts and details I have at hand than just going by rumors.

Don’t worry if you still feel a bit uncomfortable, you can always hide the story by clicking on top right of it and selecting hide. And don’t worry too much =)
PS: All content is my own opinion and the links I shared are the ones I felt were most fitting for the content. If there is any wrong information about the links feel free to share. I meant to affect no one’s sentiments but this is my opinion and I thought of sharing it. 
%d bloggers like this: