Take 4+: Presentations on ‘Elements of Neural Networks and Deep Learning’ – Parts 1-8

“Lights, camera and … action – Take 4+!”

This post includes  a rework of all presentation of ‘Elements of Neural Networks and Deep  Learning Parts 1-8 ‘ since my earlier presentations had some missing parts, omissions and some occasional errors. So I have re-recorded all the presentations.
This series of presentation will do a deep-dive  into Deep Learning networks starting from the fundamentals. The equations required for performing learning in a L-layer Deep Learning network  are derived in detail, starting from the basics. Further, the presentations also discuss multi-class classification, regularization techniques, and gradient descent optimization methods in deep networks methods. Finally the presentations also touch on how  Deep Learning Networks can be tuned.

The corresponding implementations are available in vectorized R, Python and Octave are available in my book ‘Deep Learning from first principles:Second edition- In vectorized Python, R and Octave

1. Elements of Neural Networks and Deep Learning – Part 1
This presentation introduces Neural Networks and Deep Learning. A look at history of Neural Networks, Perceptrons and why Deep Learning networks are required and concluding with a simple toy examples of a Neural Network and how they compute. This part also includes a small digression on the basics of Machine Learning and how the algorithm learns from a data set

2. Elements of Neural Networks and Deep Learning – Part 2
This presentation takes logistic regression as an example and creates an equivalent 2 layer Neural network. The presentation also takes a look at forward & backward propagation and how the cost is minimized using gradient descent


The implementation of the discussed 2 layer Neural Network in vectorized R, Python and Octave are available in my post ‘Deep Learning from first principles in Python, R and Octave – Part 1‘

3. Elements of Neural Networks and Deep Learning – Part 3
This 3rd part, discusses a primitive neural network with an input layer, output layer and a hidden layer. The neural network uses tanh activation in the hidden layer and a sigmoid activation in the output layer. The equations for forward and backward propagation are derived.


To see the implementations for the above discussed video see my post ‘Deep Learning from first principles in Python, R and Octave – Part 2

4. Elements of Neural Network and Deep Learning – Part 4
This presentation is a continuation of my 3rd presentation in which I derived the equations for a simple 3 layer Neural Network with 1 hidden layer. In this video presentation, I discuss step-by-step the derivations for a L-Layer, multi-unit Deep Learning Network, with any activation function g(z)


The implementations of L-Layer, multi-unit Deep Learning Network in vectorized R, Python and Octave are available in my post Deep Learning from first principles in Python, R and Octave – Part 3

5. Elements of Neural Network and Deep Learning – Part 5
This presentation discusses multi-class classification using the Softmax function. The detailed derivation for the Jacobian of the Softmax is discussed, and subsequently the derivative of cross-entropy loss is also discussed in detail. Finally the final set of equations for a Neural Network with multi-class classification is derived.


The corresponding implementations in vectorized R, Python and Octave are available in the following posts
a. Deep Learning from first principles in Python, R and Octave – Part 4
b. Deep Learning from first principles in Python, R and Octave – Part 5

6. Elements of Neural Networks and Deep Learning – Part 6
This part discusses initialization methods specifically like He and Xavier. The presentation also focuses on how to prevent over-fitting using regularization. Lastly the dropout method of regularization is also discussed


The corresponding implementations in vectorized R, Python and Octave of the above discussed methods are available in my post Deep Learning from first principles in Python, R and Octave – Part 6

7. Elements of Neural Networks and Deep Learning – Part 7
This presentation introduces exponentially weighted moving average and shows how this is used in different approaches to gradient descent optimization. The key techniques discussed are learning rate decay, momentum method, rmsprop and adam.

The equivalent implementations of the gradient descent optimization techniques in R, Python and Octave can be seen in my post Deep Learning from first principles in Python, R and Octave – Part 7

8. Elements of Neural Networks and Deep Learning – Part 8
This last part touches on the method to adopt while tuning hyper-parameters in Deep Learning networks

Checkout my book ‘Deep Learning from first principles: Second Edition – In vectorized Python, R and Octave’. My book starts with the implementation of a simple 2-layer Neural Network and works its way to a generic L-Layer Deep Learning Network, with all the bells and whistles. The derivations have been discussed in detail. The code has been extensively commented and included in its entirety in the Appendix sections. My book is available on Amazon as paperback ($18.99) and in kindle version($9.99/Rs449).

This concludes this series of presentations on “Elements of Neural Networks and Deep Learning’

Also
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. Introducing cricpy:A python package to analyze performances of cricketers
3. Natural language processing: What would Shakespeare say?
4. Big Data-2: Move into the big league:Graduate from R to SparkR
5. Presentation on Wireless Technologies – Part 1
6. Introducing cricketr! : An R package to analyze performances of cricketers

To see all posts click Index of posts

Pitching yorkpy…swinging away from the leg stump to IPL – Part 3

Clocks offer at best a convenient fiction They imply that time ticks steadily, predictably forward, when our experience shows that it often does the opposite: it stretches and compresses, skips a beat and doubles back.

                                 David Eagleman
                                 

Memory is the space in which a thing happens for a second time

                                 Paul Auster
      

Introduction

In this 3rd post, yorkpy, the python avatar of my R package yorkr develops more muscle. The first two posts of yorkpy were

1. Pitching yorkpy . short of good length to IPL – Part 1 This post dealt with function which perform analytics on an IPL match between any 2 IPL teams
2. Pitching yorkpy…on the middle and outside off-stump to IPL – Part 2 The second post dealt with analytics on all matches between any 2 IPL teams.

This third post deals with analyses and analytics of an IPL team in all matches against all other IPL teams. The data for yorkpy comes from Cricsheet. The data in Cricsheet are in the form of yaml files. These files have already been converted as dataframes and stored as CSV as seen in the earlier posts.You can download all the data used in this post and the previous post at yorkpyData

The signatures of yorkpy and yorkr are identical and will work in almost the same way. However there may be some unique functions in yorkr & yorkpy, based on what my thought process was on that day!

-You can clone/download the code at Github yorkpy
-This post has been published to RPubs at yorkpy-Part3
-Download this post as PDF at IPLT20-yorkpy-part3
-You can download all the data used in this post and the previous post at yorkpyData

Note: If you would like to do a similar analysis for a different set of batsman and bowlers, you can clone/download my skeleton yorkpy-template from Github (which is the R Markdown file I have used for the analysis below).

The IPL T20 functions in yorkpy are shown below

2. Get data for all T20 matches between an IPL team and all other IPL teams

We can get all IPL T20 matches between an IPL team  and all other teams using the function below. The dir parameter should point to the folder which has the IPL T20 csv files of the individual matches (see Pitching yorkpy…short of good length to IPL-Part 1). This function creates a data frame of all the IPL T20 matches between the IPL team and all other teams and and also saves the dataframe as CSV file if save=True. If save=False the dataframe is just returned and not saved.

import pandas as pd
import os
import yorkpy.analytics as yka
#dir1= "C:\\software\\cricket-package\\yorkpyPkg\\yorkpyData\\IPLConverted"
#getAllMatchesAllOpposition("Kolkata Knight Riders",dir=dir1,save=True)

3. Save data for all matches between an IPL team and all oppositions

This can be done locally using the function below. You could use this function to get combine all IPL matches of an IPL team against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
#dir1= "C:\\software\\cricket-package\\yorkpyPkg\\yorkpyData\\IPLConverted"
#saveAllMatchesAllOppositionIPLT20(dir1)

Note: In the functions below, I have randomly chosen an IPL team for the analyses. You are free to choose any IPL team for your analysis

4.Team Batsmen partnership in Twenty20 (all matches against all IPL teams – summary)

The function below computes the highest partnerships for an IPL team against all other IPL teams for e.g. the batsmen with the highest partnership from Chennai Super Kings in all matches against all other IPL teams. Any other IPL team could have also been chosen.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Chennai Super Kings-allMatchesAllOpposition.csv") 
csk_matches = pd.read_csv(path)
m=yka.teamBatsmenPartnershiAllOppnAllMatches(csk_matches,'Chennai Super Kings',report="summary")
print(m)
##         batsman  totalPartnershipRuns
## 42     SK Raina                  3699
## 28     MS Dhoni                  2986
## 25   MEK Hussey                  1768
## 24      M Vijay                  1600
## 36  S Badrinath                  1441

5. Team Batsmen partnership in Twenty20 (all matches against all IPL teams -detailed)

The function below gives the detailed breakup of partnerships for Mumbai Indian against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Mumbai Indians-allMatchesAllOpposition.csv")
mi_matches = pd.read_csv(path)
theTeam='Mumbai Indians'
m=yka.teamBatsmenPartnershiAllOppnAllMatches(mi_matches,theTeam,report="detailed", top=3)
print(m)
##        batsman  totalPartnershipRuns      non_striker  partnershipRuns
## 0    RG Sharma                3037.0        A Symonds            142.0
## 1    RG Sharma                3037.0      AC Blizzard              5.0
## 2    RG Sharma                3037.0         AJ Finch              2.0
## 3    RG Sharma                3037.0          AP Tare             32.0
## 4    RG Sharma                3037.0        AT Rayudu            566.0
## 5    RG Sharma                3037.0          BR Dunk              1.0
## 6    RG Sharma                3037.0      CJ Anderson            183.0
## 7    RG Sharma                3037.0        CM Gautam             22.0
## 8    RG Sharma                3037.0         DR Smith             50.0
## 9    RG Sharma                3037.0       GJ Maxwell              6.0
## 10   RG Sharma                3037.0         HH Gibbs            109.0
## 11   RG Sharma                3037.0        HH Pandya            105.0
## 12   RG Sharma                3037.0  Harbhajan Singh             86.0
## 13   RG Sharma                3037.0       JC Buttler            105.0
## 14   RG Sharma                3037.0     JEC Franklin             50.0
## 15   RG Sharma                3037.0       KA Pollard            633.0
## 16   RG Sharma                3037.0       KD Karthik            170.0
## 17   RG Sharma                3037.0        KH Pandya             34.0
## 18   RG Sharma                3037.0        KV Sharma             33.0
## 19   RG Sharma                3037.0      LMP Simmons            172.0
## 20   RG Sharma                3037.0       MEK Hussey             21.0
## 21   RG Sharma                3037.0       MJ Guptill             61.0
## 22   RG Sharma                3037.0   MJ McClenaghan              2.0
## 23   RG Sharma                3037.0           N Rana             25.0
## 24   RG Sharma                3037.0         PA Patel            103.0
## 25   RG Sharma                3037.0          RE Levi             25.0
## 26   RG Sharma                3037.0       SL Malinga              0.0
## 27   RG Sharma                3037.0     SR Tendulkar            208.0
## 28   RG Sharma                3037.0        SS Tiwary             27.0
## 29   RG Sharma                3037.0         TL Suman              7.0
## ..         ...                   ...              ...              ...
## 70  KA Pollard                2344.0      CJ Anderson             82.0
## 71  KA Pollard                2344.0        CM Gautam             16.0
## 72  KA Pollard                2344.0         DR Smith             10.0
## 73  KA Pollard                2344.0      DS Kulkarni             15.0
## 74  KA Pollard                2344.0        HH Pandya            158.0
## 75  KA Pollard                2344.0  Harbhajan Singh            158.0
## 76  KA Pollard                2344.0        J Suchith             26.0
## 77  KA Pollard                2344.0       JC Buttler             37.0
## 78  KA Pollard                2344.0     JEC Franklin             38.0
## 79  KA Pollard                2344.0        JP Duminy             63.0
## 80  KA Pollard                2344.0       KD Karthik             40.0
## 81  KA Pollard                2344.0        KH Pandya            111.0
## 82  KA Pollard                2344.0        KV Sharma             13.0
## 83  KA Pollard                2344.0      LMP Simmons             77.0
## 84  KA Pollard                2344.0       MEK Hussey             10.0
## 85  KA Pollard                2344.0       MG Johnson              1.0
## 86  KA Pollard                2344.0           N Rana             60.0
## 87  KA Pollard                2344.0         PA Patel             18.0
## 88  KA Pollard                2344.0          PP Ojha             12.0
## 89  KA Pollard                2344.0         R Dhawan             25.0
## 90  KA Pollard                2344.0        R McLaren             20.0
## 91  KA Pollard                2344.0        R Sathish             27.0
## 92  KA Pollard                2344.0        RG Sharma            587.0
## 93  KA Pollard                2344.0      RJ Peterson              0.0
## 94  KA Pollard                2344.0         S Dhawan             20.0
## 95  KA Pollard                2344.0       SL Malinga             14.0
## 96  KA Pollard                2344.0     SR Tendulkar             69.0
## 97  KA Pollard                2344.0        SS Tiwary             42.0
## 98  KA Pollard                2344.0         TL Suman              2.0
## 99  KA Pollard                2344.0           Z Khan              1.0
## 
## [100 rows x 4 columns]

6. Team Batsmen partnership in Twenty20 – Chart (all matches against all IPL teams)

The function below plots the partnerships of an IPL team against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Delhi Daredevils-allMatchesAllOpposition.csv")
dd_matches = pd.read_csv(path)
yka.teamBatsmenPartnershipAllOppnAllMatchesChart(dd_matches,'Delhi Daredevils', plot=True, top=4, partnershipRuns=100)

7.Team Batsmen partnership in Twenty20 – Dataframe (all matches against all IPL teams)

This function does not plot the data but returns the dataframe to the user to plot or manipulate.

Note: Many of the plots include an additional parameters for e.g. plot which is either True or False. The default value is plot=True. When plot=True the plot will be displayed. When plot=False the data frame will be returned to the user. The user can use this to create an interactive charts. The parameter top= specifies the number of top batsmen that need to be included in the chart, and partnershipRuns gives the minimum cutoff runs in partnwerships to be considered

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kochi Tuskers Kerala-allMatchesAllOpposition.csv")
ktk_matches = pd.read_csv(path)
m=yka.teamBatsmenPartnershipAllOppnAllMatchesChart(ktk_matches,'Kochi Tuskers Kerala', plot=False, top=3, partnershipRuns=100)
print(m)
##              batsman       non_striker  partnershipRuns
## 0        BB McCullum          BJ Hodge             17.0
## 1        BB McCullum  DPMD Jayawardene            160.0
## 2        BB McCullum         M Klinger             67.0
## 3        BB McCullum          PA Patel             40.0
## 4        BB McCullum         RA Jadeja             19.0
## 5        BB McCullum        VVS Laxman             41.0
## 6        BB McCullum  Y Gnaneswara Rao             13.0
## 7   DPMD Jayawardene       BB McCullum            152.0
## 8   DPMD Jayawardene          BJ Hodge             41.0
## 9   DPMD Jayawardene         KM Jadhav              4.0
## 10  DPMD Jayawardene         M Klinger             28.0
## 11  DPMD Jayawardene           OA Shah              9.0
## 12  DPMD Jayawardene          PA Patel             25.0
## 13  DPMD Jayawardene         RA Jadeja             18.0
## 14  DPMD Jayawardene          RV Gomez             10.0
## 15  DPMD Jayawardene        VVS Laxman             12.0
## 16          BJ Hodge       BB McCullum             18.0
## 17          BJ Hodge  DPMD Jayawardene             47.0
## 18          BJ Hodge         KM Jadhav              2.0
## 19          BJ Hodge           OA Shah             19.0
## 20          BJ Hodge          PA Patel             79.0
## 21          BJ Hodge         RA Jadeja             99.0
## 22          BJ Hodge          RV Gomez             21.0

8. Team batsmen versus bowler in Twenty20-Chart (all matches against all IPL teams)

The plots below provide information on how each of the top batsmen of the IPL team fared against the opposition bowlers of all other IPL teams.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Royal Challengers Bangalore-allMatchesAllOpposition.csv")
rcb_matches = pd.read_csv(path)
yka.teamBatsmenVsBowlersAllOppnAllMatches(rcb_matches,"Royal Challengers Bangalore",plot=True,top=3,runsScored=60)

9 Team batsmen versus bowler in Twenty20-Dataframe (all matches against all IPL teams)

This function provides the batting performance of an IPL team against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kings XI Punjab-allMatchesAllOpposition.csv")
kxip_matches = pd.read_csv(path)
m=yka.teamBatsmenVsBowlersAllOppnAllMatches(kxip_matches,'Kings XI Punjab',plot=False,top=2,runsScored=50)
print(m)
##        batsman            bowler  runsScored
## 0     SE Marsh        A Chandila        20.0
## 1     SE Marsh       A Choudhary         1.0
## 2     SE Marsh          A Kumble        37.0
## 3     SE Marsh          A Mishra         0.0
## 4     SE Marsh          A Mithun         9.0
## 5     SE Marsh           A Nehra        33.0
## 6     SE Marsh           A Singh         2.0
## 7     SE Marsh         A Symonds         5.0
## 8     SE Marsh         AA Chavan        19.0
## 9     SE Marsh   AA Jhunjhunwala        15.0
## 10    SE Marsh        AB Agarkar        27.0
## 11    SE Marsh          AB Dinda        31.0
## 12    SE Marsh       AB McDonald         9.0
## 13    SE Marsh         AC Thomas         1.0
## 14    SE Marsh        AD Mathews         7.0
## 15    SE Marsh        AD Russell         8.0
## 16    SE Marsh            AJ Tye         0.0
## 17    SE Marsh        AL Menaria         6.0
## 18    SE Marsh          AM Salvi         8.0
## 19    SE Marsh          AN Ahmed        16.0
## 20    SE Marsh           AS Raut         7.0
## 21    SE Marsh      Ankit Sharma         2.0
## 22    SE Marsh        Ankit Soni        11.0
## 23    SE Marsh           B Kumar        10.0
## 24    SE Marsh             B Lee         1.0
## 25    SE Marsh        BAW Mendis        11.0
## 26    SE Marsh           BB Sran         3.0
## 27    SE Marsh          BJ Hodge        18.0
## 28    SE Marsh      Basil Thampi        17.0
## 29    SE Marsh   C de Grandhomme         8.0
## ..         ...               ...         ...
## 235  DA Miller          R Sharma         7.0
## 236  DA Miller         R Tewatia         3.0
## 237  DA Miller     R Vinay Kumar        30.0
## 238  DA Miller         RA Jadeja        84.0
## 239  DA Miller         RD Chahar         3.0
## 240  DA Miller  RE van der Merwe         5.0
## 241  DA Miller  RN ten Doeschate         1.0
## 242  DA Miller          RP Singh        35.0
## 243  DA Miller       Rashid Khan         0.0
## 244  DA Miller         S Aravind         7.0
## 245  DA Miller            S Kaul        23.0
## 246  DA Miller         S Kaushik         8.0
## 247  DA Miller           S Ladda         6.0
## 248  DA Miller          S Nadeem        11.0
## 249  DA Miller          SK Raina         2.0
## 250  DA Miller        SL Malinga         9.0
## 251  DA Miller   SMSM Senanayake         6.0
## 252  DA Miller         SP Narine        10.0
## 253  DA Miller         SR Watson        16.0
## 254  DA Miller         STR Binny        14.0
## 255  DA Miller   Shakib Al Hasan         3.0
## 256  DA Miller          TA Boult        20.0
## 257  DA Miller        TG Southee        11.0
## 258  DA Miller          UT Yadav        51.0
## 259  DA Miller          VR Aaron        19.0
## 260  DA Miller          VS Malik         3.0
## 261  DA Miller         YK Pathan         0.0
## 262  DA Miller         YS Chahal        35.0
## 263  DA Miller      Yuvraj Singh        11.0
## 264  DA Miller            Z Khan         2.0
## 
## [265 rows x 3 columns]

10. Team batting scorecard(all matches against all IPL teams)

This function provides the overall scorecard for an IPL team in all matches against all other IPL teams. The batting scorecard shows the top batsmen for Kolkata Knight Riders below

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kolkata Knight Riders-allMatchesAllOpposition.csv")
kkr_matches = pd.read_csv(path)
scorecard=yka.teamBattingScorecardAllOppnAllMatches(kkr_matches,'Kolkata Knight Riders')
print(scorecard)
##              batsman    runs  balls   4s  6s          SR
## 19         G Gambhir  3035.0   2533  352  46  119.818397
## 17         YK Pathan  1893.0   1421  150  86  133.216045
## 22        RV Uthappa  1806.0   1311  200  54  137.757437
## 16         JH Kallis  1295.0   1237  128  23  104.688763
## 23         MK Pandey  1270.0   1048  103  38  121.183206
## 0         SC Ganguly  1031.0    977  105  36  105.527124
## 12         MK Tiwary  1002.0    921   86  23  108.794788
## 1        BB McCullum   882.0    754   92  32  116.976127
## 25          SA Yadav   608.0    474   54  21  128.270042
## 15          MS Bisla   543.0    518   60  16  104.826255
## 26        AD Russell   516.0    308   45  34  167.532468
## 4          DJ Hussey   511.0    417   31  28  122.541966
## 24   Shakib Al Hasan   498.0    399   44  15  124.812030
## 10          BJ Hodge   476.0    430   47  10  110.697674
## 11          CH Gayle   463.0    350   45  26  132.285714
## 18        EJG Morgan   444.0    373   45  16  119.034853
## 54           CA Lynn   378.0    250   30  23  151.200000
## 6          LR Shukla   374.0    320   31  15  116.875000
## 29  RN ten Doeschate   326.0    238   26  15  136.974790
## 21            DB Das   304.0    267   23  16  113.857678
## 3            WP Saha   298.0    213   24  12  139.906103
## 28         SP Narine   271.0    193   36  12  140.414508
## 13        AD Mathews   249.0    211   20   8  118.009479
## 33       Salman Butt   193.0    172   30   2  112.209302
## 41        MN van Wyk   167.0    135   19   1  123.703704
## 7         AB Agarkar   160.0    137   12   5  116.788321
## 20          R Bhatia   159.0    134   15   3  118.656716
## 51   C de Grandhomme   126.0     92   10   6  136.956522
## 39         CA Pujara   122.0    119   14   3  102.521008
## 40           OA Shah   115.0     96    7   5  119.791667
## ..               ...     ...    ...  ...  ..         ...
## 50         JO Holder    22.0     20    2   1  110.000000
## 65     Kuldeep Yadav    20.0     22    2   0   90.909091
## 71         BJ Haddin    18.0     11    2   1  163.636364
## 70   NM Coulter-Nile    14.0     13    0   2  107.692308
## 47          L Balaji    13.0     12    1   0  108.333333
## 55   SMSM Senanayake    10.0     17    0   0   58.823529
## 53          M Morkel     9.0      8    0   0  112.500000
## 62          AN Ghosh     7.0      8    1   0   87.500000
## 32           GB Hogg     7.0      6    0   0  116.666667
## 56        MV Boucher     6.0      6    0   0  100.000000
## 77     Azhar Mahmood     6.0      8    1   0   75.000000
## 78          DM Bravo     6.0      5    1   0  120.000000
## 68         SS Shaikh     6.0      7    1   0   85.714286
## 66          TA Boult     5.0      8    0   0   62.500000
## 76    Mohammed Shami     5.0     10    0   0   50.000000
## 80           P Dogra     5.0      8    0   0   62.500000
## 69     R Vinay Kumar     4.0      7    0   0   57.142857
## 75        AS Rajpoot     4.0      7    1   0   57.142857
## 43     Mandeep Singh     4.0     11    1   0   36.363636
## 37          AB Dinda     4.0      8    0   0   50.000000
## 79        PJ Sangwan     4.0      2    1   0  200.000000
## 73         R McLaren     3.0      6    0   0   50.000000
## 67         SB Bangar     2.0      9    0   0   22.222222
## 57       RS Gavaskar     2.0      8    0   0   25.000000
## 72     Shoaib Akhtar     2.0      8    0   0   25.000000
## 38  Mashrafe Mortaza     2.0      2    0   0  100.000000
## 63        BAW Mendis     1.0      2    0   0   50.000000
## 58           SE Bond     1.0      2    0   0   50.000000
## 44     CK Langeveldt     0.0      1    0   0    0.000000
## 30        PJ Cummins     0.0      2    0   0    0.000000
## 
## [81 rows x 6 columns]

10a. Team batting scorecard(all matches against all IPL teams)

The output below shows the Chennai Super Kings against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Chennai Super Kings-allMatchesAllOpposition.csv")
csk_matches = pd.read_csv(path)
scorecard=yka.teamBattingScorecardAllOppnAllMatches(csk_matches,'Chennai Super Kings')
print(scorecard)
##             batsman  runs  balls   4s   6s          SR
## 3          SK Raina  3699   2735  322  150  135.246801
## 5          MS Dhoni  2986   2199  218  126  135.788995
## 17       MEK Hussey  1768   1461  181   45  121.013005
## 11          M Vijay  1600   1289  141   66  124.127230
## 4       S Badrinath  1441   1245  154   28  115.742972
## 9         ML Hayden  1107    838  121   44  132.100239
## 18     F du Plessis  1081    867   92   29  124.682814
## 25         DR Smith   965    766  102   50  125.979112
## 26      BB McCullum   841    634   83   42  132.649842
## 6         JA Morkel   827    591   51   48  139.932318
## 20         DJ Bravo   706    543   54   30  130.018416
## 19        RA Jadeja   670    533   46   23  125.703565
## 0          PA Patel   516    529   67    7   97.542533
## 2        SP Fleming   196    171   27    3  114.619883
## 13         R Ashwin   190    208   19    1   91.346154
## 21         S Vidyut   145    115   21    3  126.086957
## 31          WP Saha   144    138    8    8  104.347826
## 1        S Anirudha   133    116    9    7  114.655172
## 33        DJ Hussey   116     96    8    6  120.833333
## 38           P Negi   116     77   10    5  150.649351
## 10         JDP Oram   106    107    6    5   99.065421
## 29        GJ Bailey    63     67    9    0   94.029851
## 22       A Flintoff    62     57    5    2  108.771930
## 8           MS Gony    50     39    2    5  128.205128
## 7   Joginder Sharma    36     30    1    2  120.000000
## 27         M Manhas    35     26    3    1  134.615385
## 28        MM Sharma    29     26    1    2  111.538462
## 23        SB Jakati    27     28    3    0   96.428571
## 12          JM Kemp    26     25    1    1  104.000000
## 14         L Balaji    22     35    1    1   62.857143
## 24     DE Bollinger    21     23    1    1   91.304348
## 41    CK Kapugedera    16     24    0    0   66.666667
## 37        CH Morris    14     17    0    0   82.352941
## 30       T Thushara    12     19    0    0   63.157895
## 42          M Ntini    11     19    2    0   57.894737
## 15   M Muralitharan     9     13    1    0   69.230769
## 32  KMDN Kulasekara     5      3    1    0  166.666667
## 34        SB Styris     5      2    1    0  250.000000
## 35       B Laughlin     4      9    0    0   44.444444
## 16          S Tyagi     3      4    0    0   75.000000
## 45  KB Arun Karthik     3      5    0    0   60.000000
## 36       AS Rajpoot     2      6    0    0   33.333333
## 43          RG More     2      2    0    0  100.000000
## 44         S Randiv     2      4    0    0   50.000000
## 39          A Nehra     1      7    0    0   14.285714
## 40         A Mukund     0      1    0    0    0.000000

11.Team Bowling scorecard (all matches against all IPL teams)

The output below gives the bowling performance of an IPL team against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Sunrisers Hyderabad-allMatchesAllOpposition.csv")
srh_matches = pd.read_csv(path)
scorecard=yka.teamBowlingScorecardAllOppnAllMatches(srh_matches,'Sunrisers Hyderabad')
## C:\Users\Ganesh\ANACON~1\lib\site-packages\yorkpy\analytics.py:564: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
##   df1['over']=df1.delivery.astype(int)
## C:\Users\Ganesh\ANACON~1\lib\site-packages\yorkpy\analytics.py:567: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
##   df1['runsConceded']=df1['runs'] + df1['wides'] + df1['noballs']
print(scorecard)
##               bowler  overs  runs  maidens  wicket   econrate
## 60       JP Faulkner     28   192        0      15   6.857143
## 83         MM Sharma     37   334        0      13   9.027027
## 119       SL Malinga     31   215        0      13   6.935484
## 123        SR Watson     30   281        0      13   9.366667
## 90   NM Coulter-Nile     24   166        0      12   6.916667
## 31          DJ Bravo     26   184        0      12   7.076923
## 135         UT Yadav     37   297        0      12   8.027027
## 125   Sandeep Sharma     32   280        0      11   8.750000
## 75          M Morkel     25   195        0       9   7.800000
## 81    MJ McClenaghan     24   175        0       9   7.291667
## 5           AB Dinda     23   165        0       9   7.173913
## 55        JD Unadkat     20   167        0       8   8.350000
## 36       DS Kulkarni     28   200        0       8   7.142857
## 25         CH Morris     24   190        0       7   7.916667
## 101         R Bhatia     18   128        0       7   7.111111
## 70     Kuldeep Yadav     16   129        0       7   8.062500
## 11          AR Patel     27   208        0       7   7.703704
## 122        SP Narine     43   282        0       7   6.558140
## 141        YS Chahal     26   224        0       6   8.615385
## 44   Harbhajan Singh     39   264        0       6   6.769231
## 96         PP Chawla     21   140        0       6   6.666667
## 4            A Zampa      4    19        0       6   4.750000
## 126  Shakib Al Hasan     14    99        1       6   7.071429
## 80        MG Johnson     20   155        0       6   7.750000
## 59         JP Duminy     10    80        0       5   8.000000
## 58         JO Holder     15   113        0       5   7.533333
## 92           P Kumar     23   173        0       5   7.521739
## 100         R Ashwin     28   142        0       5   5.071429
## 2           A Mishra     18   144        0       4   8.000000
## 106    R Vinay Kumar     19   154        0       4   8.105263
## ..               ...    ...   ...      ...     ...        ...
## 6     AD Mascarenhas      4    25        0       0   6.250000
## 13        Ankit Soni      2    31        0       0  15.500000
## 132          TM Head      1    11        0       0  11.000000
## 10          AN Ahmed      6    63        0       0  10.500000
## 131       TM Dilshan      1    10        0       0  10.000000
## 134     Tejas Baroka      3    33        0       0  11.000000
## 73          M Ashwin      1     6        0       0   6.000000
## 109        RG Sharma      1     5        0       0   5.000000
## 22      Basil Thampi      2    21        0       0  10.500000
## 23           C Munro      1     8        0       0   8.000000
## 68         KV Sharma      2    19        0       0   9.500000
## 77           M Vijay      4    24        0       0   6.000000
## 66         KJ Abbott      3    34        0       0  11.333333
## 65         KH Pandya      2    17        0       0   8.500000
## 82          MM Patel      3    22        0       0   7.333333
## 62          K Rabada      4    59        0       0  14.750000
## 85        MP Stoinis      3    28        0       0   9.333333
## 54         JA Morkel      3    35        0       0  11.666667
## 46          I Sharma      8    64        0       0   8.000000
## 94        PJ Cummins      4    37        0       0   9.250000
## 95        PJ Sangwan      8    82        0       0  10.250000
## 103        R Sathish      1     9        0       0   9.000000
## 38          DW Steyn      2    17        0       0   8.500000
## 108          RG More      2    28        0       0  14.000000
## 34         DJG Sammy      2    18        0       0   9.000000
## 33     DJ Muthuswami      2    20        0       0  10.000000
## 32          DJ Hooda      5    45        0       0   9.000000
## 24          CH Gayle      3    24        0       0   8.000000
## 116        SA Abbott      2    21        0       0  10.500000
## 72         LR Shukla      2    28        0       0  14.000000
## 
## [144 rows x 6 columns]

11a.Team Bowling scorecard (all matches against all IPL teams)

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Rajasthan Royals-allMatchesAllOpposition.csv")
rr_matches = pd.read_csv(path)
scorecard=yka.teamBowlingScorecardAllOppnAllMatches(rr_matches,'Rajasthan Royals')
print(scorecard)
##                bowler  overs  runs  maidens  wicket   econrate
## 2            A Mishra     63   426        0      29   6.761905
## 66          JA Morkel     38   301        0      16   7.921053
## 129     R Vinay Kumar     48   406        0      15   8.458333
## 135          RP Singh     41   255        0      14   6.219512
## 95        MF Maharoof     23   139        0      14   6.043478
## 118         PP Chawla     45   353        0      14   7.844444
## 130         RA Jadeja     32   227        0      14   7.093750
## 50           DW Steyn     43   232        0      13   5.395349
## 56    Harbhajan Singh     45   341        0      12   7.577778
## 1            A Kumble     21   108        1      12   5.142857
## 159        SL Malinga     49   363        0      12   7.408163
## 60          IK Pathan     37   279        0      11   7.540541
## 82         KA Pollard     21   201        0      11   9.571429
## 119           PP Ojha     46   426        0      11   9.260870
## 121          R Ashwin     29   222        0      11   7.655172
## 22            B Kumar     31   233        0      11   7.516129
## 3             A Nehra     32   214        0      11   6.687500
## 41           DJ Bravo     30   292        0      10   9.733333
## 110           P Kumar     48   329        1      10   6.854167
## 58           I Sharma     37   284        0       9   7.675676
## 168   Shakib Al Hasan     25   153        0       9   6.120000
## 87           L Balaji     33   277        0       9   8.393939
## 122          R Bhatia     19   121        0       8   6.368421
## 48        DS Kulkarni     21   148        0       8   7.047619
## 101         MM Sharma     20   142        0       8   7.100000
## 174          UT Yadav     25   203        0       8   8.120000
## 15           AR Patel     16   110        0       7   6.875000
## 133         RJ Harris     16   132        0       7   8.250000
## 72          JH Kallis     37   254        0       7   6.864865
## 192            Z Khan     33   213        0       7   6.454545
## ..                ...    ...   ...      ...     ...        ...
## 170      Shoaib Ahmed      2    19        0       0   9.500000
## 54          GS Sandhu      4    49        0       0  12.250000
## 139          RV Gomez      1     9        0       0   9.000000
## 163         SPD Smith      0     5        0       0        inf
## 115       PC Valthaty      3    35        0       0  11.666667
## 34        CJ Anderson      4    26        0       0   6.500000
## 81         K Upadhyay      3    29        0       0   9.666667
## 79             K Goel      1    11        0       0  11.000000
## 28          BJ Rohrer      1    12        0       0  12.000000
## 78    Joginder Sharma      2    23        0       0  11.500000
## 99          MK Tiwary      2    28        0       0  14.000000
## 26       BE Hendricks      4    57        0       0  14.250000
## 102          MR Marsh      1    10        0       0  10.000000
## 106       NL McCullum      3    22        0       0   7.333333
## 113        P Prasanth      1    18        0       0  18.000000
## 114           P Suyal      4    45        0       0  11.250000
## 46      DP Vijaykumar      1    10        0       0  10.000000
## 154         SB Styris      2    14        0       0   7.000000
## 71       JEC Franklin      3    32        0       0  10.666667
## 70          JE Taylor      3    22        0       0   7.333333
## 18       Ankit Sharma      4    33        0       0   8.250000
## 134  RN ten Doeschate      2    14        0       0   7.000000
## 16       Abdur Razzak      2    29        0       0  14.500000
## 65           J Theron      6    48        0       0   8.000000
## 146          S Narwal      2    17        0       0   8.500000
## 63            J Botha      1    19        0       0  19.000000
## 149           S Tyagi      8    65        0       0   8.125000
## 151         SB Bangar      2    20        0       0  10.000000
## 13           AM Nayar      2     7        0       0   3.500000
## 0      A Ashish Reddy      3    22        0       0   7.333333
## 
## [193 rows x 6 columns]

12. Team Bowling wicket kind -Chart (all matches against all IPL teams)

The functions compute and display the kind of wickets taken(bowled, caught, lbw etc) by an IPL team in all matches against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Gujarat Lions-allMatchesAllOpposition.csv")
gl_matches = pd.read_csv(path)
yka.teamBowlingWicketKindAllOppnAllMatches(gl_matches,'Gujarat Lions',plot=True,top=5,wickets=2)

13. Team Bowling wicket kind -Dataframe (all matches against all IPL teams)

This gives the type of wickets taken for an IPL team against all other IPL teams.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Rising Pune Supergiants-allMatchesAllOpposition.csv")
rps_matches = pd.read_csv(path)
m=yka.teamBowlingWicketKindAllOppnAllMatches(rps_matches,'Rising Pune Supergiants',plot=False,top=4,wickets=10)
print(m)
##           bowler               kind  wickets
## 0        A Nehra             caught        4
## 1        A Nehra            run out        2
## 2      MM Sharma             caught        3
## 3      MM Sharma  caught and bowled        1
## 4      MM Sharma            run out        1
## 5      SR Watson             bowled        1
## 6      SR Watson             caught        4
## 7  KW Richardson             caught        3
## 8  KW Richardson       retired hurt        1

14 Team Bowler vs Batman -Plot (all matches against all IPL teams)

The function below gives the performance of bowlers against batsmen ,in all matches against another IPL team.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Rising Pune Supergiants-allMatchesAllOpposition.csv")
rps_matches = pd.read_csv(path)
yka.teamBowlersVsBatsmenAllOppnAllMatches(rps_matches,'Rising Pune Supergiants',plot=True,top=5,runsConceded=10)

15 Team Bowler vs Batman – Dataframe (all matches against all IPL teams)

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Deccan Chargers-allMatchesAllOpposition.csv")
dc_matches = pd.read_csv(path)
m=yka.teamBowlersVsBatsmenAllOppnAllMatches(dc_matches,'Deccan Chargers',plot=False,top=2,runsConceded=30)
print(m)
##        bowler          batsman  runsConceded
## 0     P Kumar   A Ashish Reddy           6.0
## 1     P Kumar        A Symonds          15.0
## 2     P Kumar      AA Bilakhia          12.0
## 3     P Kumar  AA Jhunjhunwala           1.0
## 4     P Kumar     AC Gilchrist          20.0
## 5     P Kumar    Anirudh Singh          11.0
## 6     P Kumar         B Chipli           1.0
## 7     P Kumar         CL White          11.0
## 8     P Kumar     DB Ravi Teja          15.0
## 9     P Kumar        DJ Harris           2.0
## 10    P Kumar         DR Smith           5.0
## 11    P Kumar       FH Edwards           3.0
## 12    P Kumar         HH Gibbs          46.0
## 13    P Kumar         J Theron           0.0
## 14    P Kumar        JP Duminy           4.0
## 15    P Kumar    KC Sangakkara          15.0
## 16    P Kumar        MD Mishra           4.0
## 17    P Kumar         PA Patel           9.0
## 18    P Kumar        RG Sharma          36.0
## 19    P Kumar        RJ Harris           3.0
## 20    P Kumar         S Dhawan          37.0
## 21    P Kumar          S Sohal           6.0
## 22    P Kumar        SB Styris           6.0
## 23    P Kumar    Shahid Afridi           0.0
## 24    P Kumar         TL Suman          22.0
## 25    P Kumar       VVS Laxman           5.0
## 26    P Kumar  Y Venugopal Rao           1.0
## 27  PP Chawla   A Ashish Reddy           2.0
## 28  PP Chawla        A Symonds          35.0
## 29  PP Chawla  AA Jhunjhunwala           6.0
## 30  PP Chawla     AC Gilchrist           4.0
## 31  PP Chawla         B Chipli           8.0
## 32  PP Chawla         CL White          16.0
## 33  PP Chawla     DB Ravi Teja          30.0
## 34  PP Chawla        DJ Harris           9.0
## 35  PP Chawla        DNT Zoysa           1.0
## 36  PP Chawla         HH Gibbs          30.0
## 37  PP Chawla        JP Duminy          10.0
## 38  PP Chawla    KC Sangakkara           1.0
## 39  PP Chawla         MR Marsh           1.0
## 40  PP Chawla         PA Patel           4.0
## 41  PP Chawla         PA Reddy           8.0
## 42  PP Chawla        RG Sharma          50.0
## 43  PP Chawla         S Dhawan          33.0
## 44  PP Chawla        SB Bangar           1.0
## 45  PP Chawla         TL Suman          17.0
## 46  PP Chawla       VVS Laxman           7.0
## 47  PP Chawla  Y Venugopal Rao           3.0

16 Team Wins and Losses – Summary (all matches against all IPL teams)

The function below computes and plots the number of wins and losses between an IPL team and all other IPL teams in all matches. The summary just gives the wins, losses and ties

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Chennai Super Kings-allMatchesAllOpposition.csv")
csk_matches = pd.read_csv(path)
team1='Chennai Super Kings'
yka.plotWinLossByTeamAllOpposition(csk_matches,team1,plot="summary")

16a Team Wins and Losses – Detailed (all matches against all IPL teams)

The function below computes and plot the number of wins and losses between an IPL team and all other IPL teams in all matches. This gives a breakup of which team won against this team.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Chennai Super Kings-allMatchesAllOpposition.csv")
csk_matches = pd.read_csv(path)
team1='Chennai Super Kings'
yka.plotWinLossByTeamAllOpposition(csk_matches,team1,plot="detailed")

16b Team Wins and Losses – Summary (all matches against all IPL teams)

This plot gives the wins vs losses of MI against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Mumbai Indians-allMatchesAllOpposition.csv")
mi_matches = pd.read_csv(path)
team1='Mumbai Indians'
yka.plotWinLossByTeamAllOpposition(mi_matches,team1,plot="summary")

16c Team Wins and Losses – Detailed (all matches against all IPL teams)

The function below computes and plot the number of wins and losses between an IPL team and all other IPL teams in all matches. This gives the breakup of MI wins, losses and ties

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Mumbai Indians-allMatchesAllOpposition.csv")
mi_matches = pd.read_csv(path)
team1='Mumbai Indians'
yka.plotWinLossByTeamAllOpposition(mi_matches,team1,plot="detailed")

17 Team Wins by win type (all matches against all IPL teams)

This function shows how the win happened whether by runs or by wickets in all matches played against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Royal Challengers Bangalore-allMatchesAllOpposition.csv")
rcb_matches = pd.read_csv(path)
yka.plotWinsByRunOrWicketsAllOpposition(rcb_matches,'Royal Challengers Bangalore')

18 Team Wins by toss decision (summary) (all matches against all IPL teams)

This show how Royal Challengers Bangalore fared when it chose to field on winning the toss

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Royal Challengers Bangalore-allMatchesAllOpposition.csv")
rcb_matches = pd.read_csv(path)
yka.plotWinsbyTossDecisionAllOpposition(rcb_matches,'Royal Challengers Bangalore',tossDecision='field',plot='summary')

18a. Team Wins by toss decision (detailed) (all matches against all IPL teams)

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kings XI Punjab-allMatchesAllOpposition.csv")
kxip_matches = pd.read_csv(path)
yka.plotWinsbyTossDecisionAllOpposition(kxip_matches,'Kings XI Punjab',tossDecision='field',plot='detailed')

19 Team Wins by toss decision (summary) (all matches against all IPL teams)

This plot shows how Mumbai Indians fared when it chose to bat on winning the toss against all other IPL teams.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Delhi Daredevils-allMatchesAllOpposition.csv")
mi_rcb_matches = pd.read_csv(path)
yka.plotWinsbyTossDecisionAllOpposition(mi_rcb_matches,'Mumbai Indians',tossDecision='bat',plot='summary')

20 Team Wins by toss decision (detailed)(all matches against all IPL teams)

This plot shows how Kings X1 Punjab fared when it chose to bat on winning the toss

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kings XI Punjab-allMatchesAllOpposition.csv")
kxip_matches = pd.read_csv(path)
yka.plotWinsbyTossDecisionAllOpposition(kxip_matches,'Kings XI Punjab',tossDecision='bat',plot='detailed')

Feel free to clone/download the code from Github yorkpy

Conclusion

This post included analysis of an IPL team against all other IPL teams. You can download the data for this and the earlier posts from [yorkpyData](https://github.com/tvganesh/yorkpyData

The code can be cloned/downloaded from Github

Important note: Do check out my other posts using yorkpy at yorkpy-posts

To be continued. Watch this space!

Also see
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. My book ‘Deep Learning from first principles:Second Edition’ now on Amazon
3. Designing a Social Web Portal
4. Computer Vision: Ramblings on derivatives, histograms and contours
5. Introducing cricket package yorkr: Part 3-Foxed by flight!
6. The making of Total Control Android game

To see all posts click Index of posts

Pitching yorkpy…on the middle and outside off-stump to IPL – Part 2

When you come to a fork in the road, take it.
You’ve got to be very careful if you don’t know where you are going, because you might not get there

      Yogi Berra

Try taking his (Rahul Dravid’s) wicket in the first 15 minutes. If you can’t then only try to take the remaining wickets

      Steve Waugh
      

Introduction

This post is a follow-up to my previous post, Pitching yorkpy…short of good length to IPL-Part 1, in which I analyzed individual IPL matches. In this 2nd post I analyze the data in all matches between any 2 IPL teams, say CSK-RCB, MI-KKR or DD-RPS and so on. As I have already mentioned yorky is the python clone of my R packkage yorkr and this post is almost a mirror image of my post with yorkr namely yorkr crashes the IPL party! – Part 2. The signatures of yorkpy and yorkr are identical and will work in amost the same way. yorkpy, like yorkr, uses data from Cricsheet

You can clone/download the code at Github yorkpy
This post has been published to RPubs at yorkpy-Part2
You can download this post as PDF at IPLT20-yorkpy-part2
You can download all the data used in this post and the previous post at yorkpyData

Note: If you would like to do a similar analysis for a different set of batsman and bowlers, you can clone/download my skeleton yorkpy-template from Github (which is the R Markdown file I have used for the analysis below).

2. Get data for all T20 matches between 2 teams

We can get all IPL T20 matches between any 2 teams using the function below. The dir parameter should point to the folder which has the IPL T20 csv files of the individual matches (see Pitching yorkpy…short of good length to IPL-Part 1). This function creates a data frame of all the IPL T20 matches and and also saves the dataframe as CSV file if save=True. If save=False the dataframe is just returned and not saved.

import pandas as pd
import os
import yorkpy.analytics as yka
#dir1= "C:\\software\\cricket-package\\yorkpyPkg\\yorkpyData\\IPLConverted"
#yka.getAllMatchesBetweenTeams("Kolkata Knight Riders","Delhi Daredevils",dir=dir1,save=True)

3. Save data for all matches between all combination of 2 teams

This can be done locally using the function below. You could use this function to combine all IPL Twenty20 matches between any 2 IPL teams into a single dataframe and save it in the current folder. All the dataframes for all combinations have already been done and are available as CSV files in Github at yorkpyData

import pandas as pd
import os
import yorkpy.analytics as yka
#dir1= "C:\\software\\cricket-package\\yorkpyPkg\\yorkpyData\\IPLConverted"
#yka.saveAllMatchesBetween2IPLTeams(dir1)

Note: In the functions below, I have randomly chosen any 2 IPL teams and analyze how the teams have performed against each other in different areas. You are free to choose any 2 combination of IPL teams for your analysis

4.Team Batsmen partnership in Twenty20 (all matches with opposing IPL team – summary)

The function below computes the highest partnerships between the 2 IPL teams Chennai Superkings and Delhi Daredevils. Any other 2 IPL team could have also been chosen. The summary gives the top 3 batsmen for Delhi Daredevils namely Sehwag, Gambhir and Dinesh Karthik when the report=‘summary’

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Chennai Super Kings-Delhi Daredevils-allMatches.csv")
csk_dd_matches = pd.read_csv(path)
m=yka.teamBatsmenPartnershiOppnAllMatches(csk_dd_matches,'Delhi Daredevils',report="summary")
print(m)
##            batsman  totalPartnershipRuns
## 49        V Sehwag                   233
## 12       G Gambhir                   200
## 21      KD Karthik                   180
## 10       DA Warner                   134
## 4   AB de Villiers                   133

5. Team Batsmen partnership in Twenty20 (all matches with opposing IPL team -detailed)

The function below gives the detailed breakup of partnerships between Deccan Chargers and Mumbai Indians for Deccan Chargers.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Deccan Chargers-Mumbai Indians-allMatches.csv")
dc_mi_matches = pd.read_csv(path)
theTeam='Deccan Chargers'
m=yka.teamBatsmenPartnershiOppnAllMatches(dc_mi_matches,theTeam,report="detailed", top=4)
print(m)
##          batsman  totalPartnershipRuns      non_striker  partnershipRuns
## 0   AC Gilchrist                   201        A Symonds                0
## 1   AC Gilchrist                   201         HH Gibbs               53
## 2   AC Gilchrist                   201        MD Mishra                0
## 3   AC Gilchrist                   201        RG Sharma               20
## 4   AC Gilchrist                   201    Shahid Afridi                6
## 5   AC Gilchrist                   201         TL Suman                7
## 6   AC Gilchrist                   201       VVS Laxman              115
## 7       S Dhawan                   122         A Mishra                9
## 8       S Dhawan                   122         B Chipli                1
## 9       S Dhawan                   122         CL White                2
## 10      S Dhawan                   122     DT Christian               52
## 11      S Dhawan                   122         IR Jaggi                2
## 12      S Dhawan                   122        JP Duminy                9
## 13      S Dhawan                   122    KC Sangakkara               16
## 14      S Dhawan                   122         PA Patel               22
## 15      S Dhawan                   122          S Sohal                9
## 16     RG Sharma                   103        A Symonds               11
## 17     RG Sharma                   103     AC Gilchrist               18
## 18     RG Sharma                   103         DR Smith                6
## 19     RG Sharma                   103         HH Gibbs                3
## 20     RG Sharma                   103   Jaskaran Singh               15
## 21     RG Sharma                   103        KAJ Roach                4
## 22     RG Sharma                   103        LPC Silva                0
## 23     RG Sharma                   103         TL Suman               14
## 24     RG Sharma                   103  Y Venugopal Rao               32
## 25      HH Gibbs                   102     AC Gilchrist               40
## 26      HH Gibbs                   102         DR Smith               24
## 27      HH Gibbs                   102        MD Mishra               27
## 28      HH Gibbs                   102        RG Sharma                8
## 29      HH Gibbs                   102       VVS Laxman                1
## 30      HH Gibbs                   102  Y Venugopal Rao                2

6. Team Batsmen partnership in Twenty20 – Chart (all matches with opposing IPL team)

The function below plots the partnerships in all matches between 2 IPL teams and plots as chart

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Gujarat Lions-Kings XI Punjab-allMatches.csv")
gl_kxip_matches = pd.read_csv(path)
yka.teamBatsmenPartnershipOppnAllMatchesChart(gl_kxip_matches,'Kings XI Punjab','Gujarat Lions', plot=True, top=4, partnershipRuns=20)

7.Team Batsmen partnership in Twenty20 – Dataframe (all matches with opposing IPL team)

This function does not plot the data but returns the dataframe to the user to plot or manipulate.

Note: Many of the plots include an additional parameters for e.g. plot which is either True or False. The default value is plot=True. When plot=True the plot will be displayed. When plot=False the data frame will be returned to the user. The user can use this to create an interactive charts. The parameter top= specifies the number of top batsmen that need to be included in the chart, and partnershipRuns gives the minimum cutoff runs in partnerships to be considered

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Kolkata Knight Riders-Rising Pune Supergiants-allMatches.csv")
kkr_rps_matches = pd.read_csv(path)
m=yka.teamBatsmenPartnershipOppnAllMatchesChart(kkr_rps_matches,'Rising Pune Supergiants','Kolkata Knight Riders', plot=False, top=5, partnershipRuns=20)
print(m)
##         batsman   non_striker  partnershipRuns
## 0     AM Rahane  F du Plessis               20
## 1     AM Rahane     JA Morkel               16
## 2     AM Rahane   NLTC Perera                6
## 3     AM Rahane     SPD Smith               25
## 4     AM Rahane    UT Khawaja                2
## 5     GJ Bailey     IK Pathan                4
## 6     GJ Bailey     SS Tiwary               28
## 7     GJ Bailey    UT Khawaja                1
## 8      MS Dhoni     IK Pathan                5
## 9      MS Dhoni     JA Morkel                1
## 10     MS Dhoni   NLTC Perera                2
## 11     MS Dhoni      R Ashwin                1
## 12     MS Dhoni      R Bhatia               22
## 13    SPD Smith     AM Rahane               31
## 14  NLTC Perera     AM Rahane               12
## 15  NLTC Perera      MS Dhoni               13

8. Team batsmen versus bowler in Twenty20-Chart (all matches with opposing IPL team)

The plots below provide information on how each of the top batsmen of the IPL teams fared against the opposition bowlers

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Rajasthan Royals-Royal Challengers Bangalore-allMatches.csv")
rr_rcb_matches = pd.read_csv(path)
yka.teamBatsmenVsBowlersOppnAllMatches(rr_rcb_matches,'Rajasthan Royals',"Royal Challengers Bangalore",plot=True,top=3,runsScored=20)

9 Team batsmen versus bowler in Twenty20-Dataframe (all matches with opposing IPL team)

This function provides the bowling performance, the number of overs bowled, maidens, runs conceded. wickets taken and economy rate for the IPL match

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Mumbai Indians-Delhi Daredevils-allMatches.csv")
mi_dd_matches = pd.read_csv(path)
m=yka.teamBatsmenVsBowlersOppnAllMatches(mi_dd_matches,'Delhi Daredevils',"Mumbai Indians",plot=False,top=2,runsScored=50)
print(m)
##       batsman           bowler  runsScored
## 0    V Sehwag          A Nehra         6.0
## 1    V Sehwag       AG Murtaza         6.0
## 2    V Sehwag         AM Nayar        14.0
## 3    V Sehwag         CJ McKay        10.0
## 4    V Sehwag     CRD Fernando         9.0
## 5    V Sehwag         DJ Bravo         9.0
## 6    V Sehwag      DJ Thornely         0.0
## 7    V Sehwag         DR Smith        13.0
## 8    V Sehwag      DS Kulkarni        20.0
## 9    V Sehwag  Harbhajan Singh        54.0
## 10   V Sehwag        JJ Bumrah        19.0
## 11   V Sehwag       KA Pollard        37.0
## 12   V Sehwag         MM Patel        27.0
## 13   V Sehwag          PP Ojha         7.0
## 14   V Sehwag         R Shukla         9.0
## 15   V Sehwag      RJ Peterson         7.0
## 16   V Sehwag         RP Singh        28.0
## 17   V Sehwag       SL Malinga        32.0
## 18   V Sehwag       SM Pollock        25.0
## 19   V Sehwag    ST Jayasuriya        29.0
## 20   V Sehwag           Z Khan        14.0
## 21  JP Duminy      CJ Anderson         3.0
## 22  JP Duminy        HH Pandya         7.0
## 23  JP Duminy  Harbhajan Singh        29.0
## 24  JP Duminy        J Suchith         5.0
## 25  JP Duminy        JJ Bumrah        70.0
## 26  JP Duminy       KA Pollard        29.0
## 27  JP Duminy        KH Pandya         8.0
## 28  JP Duminy       M de Lange         6.0
## 29  JP Duminy   MJ McClenaghan        14.0
## 30  JP Duminy           N Rana         1.0
## 31  JP Duminy          PP Ojha        16.0
## 32  JP Duminy    R Vinay Kumar        18.0
## 33  JP Duminy        RG Sharma         3.0
## 34  JP Duminy          S Gopal         8.0
## 35  JP Duminy       SL Malinga         8.0
## 36  JP Duminy       TG Southee         3.0

10. Team batting scorecard(all matches with opposing IPL team)

This function provides the overall scorecard for an IPL team in all matches against another IPL team. In the snippet below the batting scorecard of RCB is show against CSK. Kohli, Gayle and De villiers lead the pack.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Royal Challengers Bangalore-Chennai Super Kings-allMatches.csv")
rcb_csk_matches = pd.read_csv(path)
scorecard=yka.teamBattingScorecardOppnAllMatches(rcb_csk_matches,'Royal Challengers Bangalore',"Chennai Super Kings")
print(scorecard)
##              batsman  runs  balls  4s  6s          SR
## 5            V Kohli   706    570  51  30  123.859649
## 20          CH Gayle   270    228  12  23  118.421053
## 19    AB de Villiers   241    157  26   9  153.503185
## 6           R Dravid   133    117  18   0  113.675214
## 3          JH Kallis   123    113  21   0  108.849558
## 22        MA Agarwal   120    104  15   4  115.384615
## 2        LRPL Taylor   117    102   5   6  114.705882
## 11        RV Uthappa   115     77   7   8  149.350649
## 21         SS Tiwary    86     88   4   3   97.727273
## 17         MK Pandey    73     72  10   0  101.388889
## 32        KD Karthik    61     58   9   0  105.172414
## 34           D Wiese    51     43   4   2  118.604651
## 33           SN Khan    50     36   5   1  138.888889
## 1           W Jaffer    50     36   5   2  138.888889
## 7            P Kumar    39     25   2   2  156.000000
## 28      Yuvraj Singh    38     33   2   1  115.151515
## 4         MV Boucher    37     33   4   1  112.121212
## 23     LA Pomersbach    31     21   2   2  147.619048
## 8             Z Khan    29     27   3   0  107.407407
## 12      KP Pietersen    23     15   2   1  153.333333
## 38          CL White    21     13   2   1  161.538462
## 26       YV Takawale    19     17   4   0  111.764706
## 31          MS Bisla    17     14   3   0  121.428571
## 14     R Vinay Kumar    17     10   1   1  170.000000
## 25        RR Rossouw    15     13   1   1  115.384615
## 40        AUK Pathan    14      6   2   1  233.333333
## 42   JJ van der Wath    14     11   1   1  127.272727
## 27            VH Zol    13     12   0   1  108.333333
## 30          MA Starc    13     16   1   0   81.250000
## 24      MC Henriques    12      4   3   0  300.000000
## 44          A Mithun    11      8   2   0  137.500000
## 50          PA Patel    10     14   2   0   71.428571
## 36        SP Goswami    10     19   1   0   52.631579
## 0           B Chipli     8     12   1   0   66.666667
## 9            B Akhil     8     12   1   0   66.666667
## 29            S Rana     6      8   0   0   75.000000
## 16  RE van der Merwe     5     12   0   0   41.666667
## 49   KB Arun Karthik     5      5   0   0  100.000000
## 54     Mandeep Singh     4      7   0   0   57.142857
## 37     Misbah-ul-Haq     4      6   0   0   66.666667
## 52      NJ Maddinson     4      7   1   0   57.142857
## 51          AN Ahmed     4      1   1   0  400.000000
## 15          A Kumble     3      6   0   0   50.000000
## 43        DL Vettori     3      4   0   0   75.000000
## 47      DT Christian     2      2   0   0  100.000000
## 45   J Syed Mohammad     2      3   0   0   66.666667
## 35          HV Patel     2      5   0   0   40.000000
## 41         CA Pujara     2      6   0   0   33.333333
## 10          DW Steyn     1      5   0   0   20.000000
## 18        EJG Morgan     1      4   0   0   25.000000
## 46        RR Bhatkal     0      2   0   0    0.000000
## 48         R Rampaul     0      6   0   0    0.000000
## 13         R Bishnoi     0      1   0   0    0.000000
## 39        TM Dilshan     0      1   0   0    0.000000
## 53     Iqbal Abdulla     0      3   0   0    0.000000
## 55         S Aravind     0      1   0   0    0.000000

11.Team Bowling scorecard (all matches with opposing IPL team)

The output below gives the performance of Rajasthan Royals bowlers against Kolkata Knight Riders in all matches between the 2 IPL teams.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Kolkata Knight Riders-Rajasthan Royals-allMatches.csv")
rcb_csk_matches = pd.read_csv(path)
scorecard=yka.teamBowlingScorecardOppnAllMatches(rcb_csk_matches,'Rajasthan Royals',"Kolkata Knight Riders")
print(scorecard)
##               bowler  overs  runs  maidens  wicket   econrate
## 31   Shakib Al Hasan     25   153        0       9   6.120000
## 12          I Sharma     15   118        0       6   7.866667
## 33          Umar Gul      8    61        0       6   7.625000
## 29         SP Narine     24   155        0       6   6.458333
## 1           AB Dinda     20   126        0       6   6.300000
## 23     R Vinay Kumar      8    72        0       5   9.000000
## 22          R Bhatia     15   104        0       5   6.933333
## 0         AB Agarkar     12   105        0       4   8.750000
## 17         LR Shukla     12    87        0       4   7.250000
## 6              B Lee     15    90        0       4   6.000000
## 3         AD Russell      7    59        0       4   8.428571
## 34         YK Pathan      8    61        0       4   7.625000
## 14        JD Unadkat      4    26        0       3   6.500000
## 15         JH Kallis     20   149        0       3   7.450000
## 16          L Balaji     11    73        0       3   6.636364
## 27           SE Bond      8    52        1       3   6.500000
## 10     CK Langeveldt      4    15        0       3   3.750000
## 13     Iqbal Abdulla     10    70        0       3   7.000000
## 28   SMSM Senanayake      4    26        0       2   6.500000
## 7         BAW Mendis      4    19        0       2   4.750000
## 18          M Kartik      8    56        0       2   7.000000
## 4      Anureet Singh      4    35        0       2   8.750000
## 32          UT Yadav      7    67        0       2   9.571429
## 30         SS Sarkar      3    15        0       1   5.000000
## 26        SC Ganguly      6    61        0       1  10.166667
## 5      Azhar Mahmood      3    41        0       1  13.666667
## 19          M Morkel      8    78        0       1   9.750000
## 11         DJ Hussey      2    26        0       0  13.000000
## 2         AD Mathews      3    33        0       0  11.000000
## 8           BJ Hodge      2    34        0       0  17.000000
## 25          S Narwal      2    17        0       0   8.500000
## 24  RN ten Doeschate      2    14        0       0   7.000000
## 21         PP Chawla      4    39        0       0   9.750000
## 20    Mohammed Shami      3    26        0       0   8.666667
## 9           CH Gayle      4    20        0       0   5.000000

12. Team Bowling wicket kind -Chart (all matches with opposing IPL team)

The functions compute and display the kind of wickets taken(bowled, caught, lbw etc) by an IPL team in all matches against another IPL team

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Chennai Super Kings-Rajasthan Royals-allMatches.csv")
csk_rr_matches = pd.read_csv(path)
yka.teamBowlingWicketKindOppositionAllMatches(csk_rr_matches,'Chennai Super Kings','Rajasthan Royals',plot=True,top=5,wickets=1)

13. Team Bowling wicket kind -Dataframe (all matches with opposing IPL team)

This gives the type of wickets taken

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Delhi Daredevils-Pune Warriors-allMatches.csv")
dd_pw_matches = pd.read_csv(path)
m=yka.teamBowlingWicketKindOppositionAllMatches(dd_pw_matches,'Pune Warriors','Delhi Daredevils',plot=False,top=4,wickets=1)
print(m)
##       bowler    kind  wickets
## 0  IK Pathan  bowled        1
## 1  IK Pathan  caught        3
## 2   M Morkel  bowled        1
## 3   M Morkel  caught        3
## 4   S Nadeem  bowled        1
## 5   S Nadeem  caught        2
## 6   UT Yadav  caught        3

14 Team Bowler vs Batman -Plot (all matches with opposing IPL team)

The function below gives the performance of bowlers in all matches against another IPL team.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Sunrisers Hyderabad-Kolkata Knight Riders-allMatches.csv")
srh_kkr_matches = pd.read_csv(path)
yka.teamBowlersVsBatsmenOppnAllMatches(srh_kkr_matches,'Sunrisers Hyderabad','Kolkata Knight Riders',plot=True,top=5,runsConceded=10)

15 Team Bowler vs Batman – Dataframe (all matches with opposing IPL team)

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Royal Challengers Bangalore-Kings XI Punjab-allMatches.csv")
srh_kkr_matches = pd.read_csv(path)
m=yka.teamBowlersVsBatsmenOppnAllMatches(srh_kkr_matches,'Royal Challengers Bangalore','Kings XI Punjab',plot=False,top=1,runsConceded=30)
print(m)
##        bowler           batsman  runsConceded
## 0   PP Chawla          A Kumble             1
## 1   PP Chawla          A Mithun             1
## 2   PP Chawla       AB McDonald             3
## 3   PP Chawla    AB de Villiers            29
## 4   PP Chawla         CA Pujara            13
## 5   PP Chawla          CH Gayle            62
## 6   PP Chawla     CK Langeveldt             1
## 7   PP Chawla          CL White             3
## 8   PP Chawla        DL Vettori             1
## 9   PP Chawla          DT Patil             4
## 10  PP Chawla         JH Kallis            17
## 11  PP Chawla   JJ van der Wath             1
## 12  PP Chawla   KB Arun Karthik             4
## 13  PP Chawla      KP Pietersen            14
## 14  PP Chawla       LRPL Taylor             6
## 15  PP Chawla            M Kaif             2
## 16  PP Chawla         MK Pandey            10
## 17  PP Chawla        MV Boucher             9
## 18  PP Chawla     Misbah-ul-Haq             0
## 19  PP Chawla           P Kumar             0
## 20  PP Chawla          R Dravid            28
## 21  PP Chawla  RE van der Merwe             7
## 22  PP Chawla        RV Uthappa            19
## 23  PP Chawla         SS Tiwary             6
## 24  PP Chawla           V Kohli            56
## 25  PP Chawla            Z Khan             0

16 Team Wins and Losses (all matches with opposing IPL team)

The function below computes and plot the number of wins and losses in a head-on confrontation between 2 IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Chennai Super Kings-Delhi Daredevils-allMatches.csv")
csk_dd_matches = pd.read_csv(path)
yka.plotWinLossBetweenTeams(csk_dd_matches,'Chennai Super Kings','Delhi Daredevils')

17 Team Wins by win type (all matches with opposing IPL team)

This function shows how the win happened whether by runs or by wickets

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Chennai Super Kings-Delhi Daredevils-allMatches.csv")
csk_dd_matches = pd.read_csv(path)
yka.plotWinsByRunOrWickets(csk_dd_matches,'Chennai Super Kings')

18 Team Wins by toss decision-field (all matches with opposing IPL team)

This show how Rajasthan Royals fared when it chose to field on winning the toss

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Rajasthan Royals-Kings XI Punjab-allMatches.csv")
rr_kxip_matches = pd.read_csv(path)
yka.plotWinsbyTossDecision(rr_kxip_matches,'Rajasthan Royals',tossDecision='field')

18 Team Wins by toss decision-bat (all matches with opposing IPL team)

This plot shows how Mumbai Indians fared when it chose to bat on winning the toss

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Mumbai Indians-Royal Challengers Bangalore-allMatches.csv")
mi_rcb_matches = pd.read_csv(path)
yka.plotWinsbyTossDecision(mi_rcb_matches,'Mumbai Indians',tossDecision='bat')

Feel free to clone/download the code from Github yorkpy

Important note: Do check out my other posts using yorkpy at yorkpy-posts

My presentations on ‘Elements of Neural Networks & Deep Learning’ -Parts 6,7,8

This is the final set of presentations in my series ‘Elements of Neural Networks and Deep Learning’. This set follows the earlier 2 sets of presentations namely
1. My presentations on ‘Elements of Neural Networks & Deep Learning’ -Part1,2,3
2. My presentations on ‘Elements of Neural Networks & Deep Learning’ -Parts 4,5

In this final set of presentations I discuss initialization methods, regularization techniques including dropout. Next I also discuss gradient descent optimization methods like momentum, rmsprop, adam etc. Lastly, I briefly also touch on hyper-parameter tuning approaches. The corresponding implementations are available in vectorized R, Python and Octave are available in my book ‘Deep Learning from first principles:Second edition- In vectorized Python, R and Octave

1. Elements of Neural Networks and Deep Learning – Part 6
This part discusses initialization methods specifically like He and Xavier. The presentation also focuses on how to prevent over-fitting using regularization. Lastly the dropout method of regularization is also discusses


The corresponding implementations in vectorized R, Python and Octave of the above discussed methods are available in my post Deep Learning from first principles in Python, R and Octave – Part 6

2. Elements of Neural Networks and Deep Learning – Part 7
This presentation introduces exponentially weighted moving average and shows how this is used in different approaches to gradient descent optimization. The key techniques discussed are learning rate decay, momentum method, rmsprop and adam.


The equivalent implementations of the gradient descent optimization techniques in R, Python and Octave can be seen in my post Deep Learning from first principles in Python, R and Octave – Part 7

3. Elements of Neural Networks and Deep Learning – Part 8
This last part touches upon hyper-parameter tuning in Deep Learning networks


This concludes this series of presentations on “Elements of Neural Networks and Deep Learning’

Important note: Do check out my later version of these videos at Take 4+: Presentations on ‘Elements of Neural Networks and Deep Learning’ – Parts 1-8 . These have more content and also include some corrections. Check it out!

Checkout my book ‘Deep Learning from first principles: Second Edition – In vectorized Python, R and Octave’. My book starts with the implementation of a simple 2-layer Neural Network and works its way to a generic L-Layer Deep Learning Network, with all the bells and whistles. The derivations have been discussed in detail. The code has been extensively commented and included in its entirety in the Appendix sections. My book is available on Amazon as paperback ($18.99) and and in kindle version($9.99/Rs449).

See also
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. Big Data-1: Move into the big league:Graduate from Python to Pyspark
3. My travels through the realms of Data Science, Machine Learning, Deep Learning and (AI)
4. Revisiting crimes against women in India
5. Introducing cricket package yorkr: Part 1- Beaten by sheer pace!
6. Deblurring with OpenCV: Weiner filter reloaded
7. Taking a closer look at Quantum gates and their operations

To see all posts click Index of posts

My presentations on ‘Elements of Neural Networks & Deep Learning’ -Parts 4,5

This is the next set of presentations on “Elements of Neural Networks and Deep Learning”.  In the 4th presentation I discuss and derive the generalized equations for a multi-unit, multi-layer Deep Learning network.  The 5th presentation derives the equations for a Deep Learning network when performing multi-class classification along with the derivations for cross-entropy loss. The corresponding implementations are available in vectorized R, Python and Octave are available in my book ‘Deep Learning from first principles:Second edition- In vectorized Python, R and Octave

Important note: Do check out my later version of these videos at Take 4+: Presentations on ‘Elements of Neural Networks and Deep Learning’ – Parts 1-8 . These have more content and also include some corrections. Check it out!

1. Elements of Neural Network and Deep Learning – Part 4
This presentation is a continuation of my 3rd presentation in which I derived the equations for a simple 3 layer Neural Network with 1 hidden layer. In this video presentation, I discuss step-by-step the derivations for a L-Layer, multi-unit Deep Learning Network, with any activation function g(z)


The implementations of L-Layer, multi-unit Deep Learning Network in vectorized R, Python and Octave are available in my post Deep Learning from first principles in Python, R and Octave – Part 3

2. Elements of Neural Network and Deep Learning – Part 5
This presentation discusses multi-class classification using the Softmax function. The detailed derivation for the Jacobian of the Softmax is discussed, and subsequently the derivative of cross-entropy loss is also discussed in detail. Finally the final set of equations for a Neural Network with multi-class classification is derived.


The corresponding implementations in vectorized R, Python and Octave are available in the following posts
a. Deep Learning from first principles in Python, R and Octave – Part 4
b. Deep Learning from first principles in Python, R and Octave – Part 5

To be continued. Watch this space!

Checkout my book ‘Deep Learning from first principles: Second Edition – In vectorized Python, R and Octave’. My book starts with the implementation of a simple 2-layer Neural Network and works its way to a generic L-Layer Deep Learning Network, with all the bells and whistles. The derivations have been discussed in detail. The code has been extensively commented and included in its entirety in the Appendix sections. My book is available on Amazon as paperback ($18.99) and in kindle version($9.99/Rs449).

Also see
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. Big Data-2: Move into the big league:Graduate from R to SparkR
3. Introducing QCSimulator: A 5-qubit quantum computing simulator in R
4. My TEDx talk on the “Internet of Things
5. Rock N’ Roll with Bluemix, Cloudant & NodeExpress
6. GooglyPlus: yorkr analyzes IPL players, teams, matches with plots and tables
7. Literacy in India – A deepR dive
8. Fun simulation of a Chain in Android

To see all posts click Index of Posts

My presentations on ‘Elements of Neural Networks & Deep Learning’ -Part1,2,3

I will be uploading a series of presentations on ‘Elements of Neural Networks and Deep Learning’. In these video presentations I discuss the derivations of L -Layer Deep Learning Networks, starting from the basics. The corresponding implementations are available in vectorized R, Python and Octave are available in my book ‘Deep Learning from first principles:Second edition- In vectorized Python, R and Octave

1. Elements of Neural Networks and Deep Learning – Part 1
This presentation introduces Neural Networks and Deep Learning. A look at history of Neural Networks, Perceptrons and why Deep Learning networks are required and concluding with a simple toy examples of a Neural Network and how they compute

2. Elements of Neural Networks and Deep Learning – Part 2
This presentation takes logistic regression as an example and creates an equivalent 2 layer Neural network. The presentation also takes a look at forward & backward propagation and how the cost is minimized using gradient descent


The implementation of the discussed 2 layer Neural Network in vectorized R, Python and Octave are available in my post ‘Deep Learning from first principles in Python, R and Octave – Part 1

3. Elements of Neural Networks and Deep Learning – Part 3
This 3rd part, discusses a primitive neural network with an input layer, output layer and a hidden layer. The neural network uses tanh activation in the hidden layer and a sigmoid activation in the output layer. The equations for forward and backward propagation are derived.


To see the implementations for the above discussed video see my post ‘Deep Learning from first principles in Python, R and Octave – Part 2

Important note: Do check out my later version of these videos at Take 4+: Presentations on ‘Elements of Neural Networks and Deep Learning’ – Parts 1-8 . These have more content and also include some corrections. Check it out!

To be continued. Watch this space!

Checkout my book ‘Deep Learning from first principles: Second Edition – In vectorized Python, R and Octave’. My book starts with the implementation of a simple 2-layer Neural Network and works its way to a generic L-Layer Deep Learning Network, with all the bells and whistles. The derivations have been discussed in detail. The code has been extensively commented and included in its entirety in the Appendix sections. My book is available on Amazon as paperback ($18.99) and in kindle version($9.99/Rs449).

You may also like
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. Introducing cricpy:A python package to analyze performances of cricketers
3. Natural language processing: What would Shakespeare say?
4. TWS-4: Gossip protocol: Epidemics and rumors to the rescue
5. Getting started with memcached-libmemcached
6. Simplifying ML: Impact of degree of polynomial degree on bias & variance and other insights

To see all posts click Index of posts

My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon

Are you wondering whether to get into the ‘R’ bus or ‘Python’ bus?
My suggestion is to you is “Why not get into the ‘R and Python’ train?”

The third edition of my book ‘Practical Machine Learning with R and Python – Machine Learning in stereo’ is now available in both paperback ($12.99) and kindle ($8.99/Rs449) versions.  In the third edition all code sections have been re-formatted to use the fixed width font ‘Consolas’. This neatly organizes output which have columns like confusion matrix, dataframes etc to be columnar, making the code more readable.  There is a science to formatting too!! which improves the look and feel. It is little wonder that Steve Jobs had a keen passion for calligraphy! Additionally some typos have been fixed.

 

In this book I implement some of the most common, but important Machine Learning algorithms in R and equivalent Python code.
1. Practical machine with R and Python: Third Edition – Machine Learning in Stereo(Paperback-$12.99)
2. Practical machine with R and Python Third Edition – Machine Learning in Stereo(Kindle- $8.99/Rs449)

This book is ideal both for beginners and the experts in R and/or Python. Those starting their journey into datascience and ML will find the first 3 chapters useful, as they touch upon the most important programming constructs in R and Python and also deal with equivalent statements in R and Python. Those who are expert in either of the languages, R or Python, will find the equivalent code ideal for brushing up on the other language. And finally,those who are proficient in both languages, can use the R and Python implementations to internalize the ML algorithms better.

Here is a look at the topics covered

Table of Contents
Preface …………………………………………………………………………….4
Introduction ………………………………………………………………………6
1. Essential R ………………………………………………………………… 8
2. Essential Python for Datascience ……………………………………………57
3. R vs Python …………………………………………………………………81
4. Regression of a continuous variable ……………………………………….101
5. Classification and Cross Validation ………………………………………..121
6. Regression techniques and regularization ………………………………….146
7. SVMs, Decision Trees and Validation curves ………………………………191
8. Splines, GAMs, Random Forests and Boosting ……………………………222
9. PCA, K-Means and Hierarchical Clustering ………………………………258
References ……………………………………………………………………..269

Pick up your copy today!!
Hope you have a great time learning as I did while implementing these algorithms!