Take 4+: Presentations on ‘Elements of Neural Networks and Deep Learning’ – Parts 1-8


“Lights, camera and … action – Take 4+!”

This post includes  a rework of all presentation of ‘Elements of Neural Networks and Deep  Learning Parts 1-8 ‘ since my earlier presentations had some missing parts, omissions and some occasional errors. So I have re-recorded all the presentations.
This series of presentation will do a deep-dive  into Deep Learning networks starting from the fundamentals. The equations required for performing learning in a L-layer Deep Learning network  are derived in detail, starting from the basics. Further, the presentations also discuss multi-class classification, regularization techniques, and gradient descent optimization methods in deep networks methods. Finally the presentations also touch on how  Deep Learning Networks can be tuned.

The corresponding implementations are available in vectorized R, Python and Octave are available in my book ‘Deep Learning from first principles:Second edition- In vectorized Python, R and Octave

1. Elements of Neural Networks and Deep Learning – Part 1
This presentation introduces Neural Networks and Deep Learning. A look at history of Neural Networks, Perceptrons and why Deep Learning networks are required and concluding with a simple toy examples of a Neural Network and how they compute. This part also includes a small digression on the basics of Machine Learning and how the algorithm learns from a data set

2. Elements of Neural Networks and Deep Learning – Part 2
This presentation takes logistic regression as an example and creates an equivalent 2 layer Neural network. The presentation also takes a look at forward & backward propagation and how the cost is minimized using gradient descent


The implementation of the discussed 2 layer Neural Network in vectorized R, Python and Octave are available in my post ‘Deep Learning from first principles in Python, R and Octave – Part 1‘

3. Elements of Neural Networks and Deep Learning – Part 3
This 3rd part, discusses a primitive neural network with an input layer, output layer and a hidden layer. The neural network uses tanh activation in the hidden layer and a sigmoid activation in the output layer. The equations for forward and backward propagation are derived.


To see the implementations for the above discussed video see my post ‘Deep Learning from first principles in Python, R and Octave – Part 2

4. Elements of Neural Network and Deep Learning – Part 4
This presentation is a continuation of my 3rd presentation in which I derived the equations for a simple 3 layer Neural Network with 1 hidden layer. In this video presentation, I discuss step-by-step the derivations for a L-Layer, multi-unit Deep Learning Network, with any activation function g(z)


The implementations of L-Layer, multi-unit Deep Learning Network in vectorized R, Python and Octave are available in my post Deep Learning from first principles in Python, R and Octave – Part 3

5. Elements of Neural Network and Deep Learning – Part 5
This presentation discusses multi-class classification using the Softmax function. The detailed derivation for the Jacobian of the Softmax is discussed, and subsequently the derivative of cross-entropy loss is also discussed in detail. Finally the final set of equations for a Neural Network with multi-class classification is derived.


The corresponding implementations in vectorized R, Python and Octave are available in the following posts
a. Deep Learning from first principles in Python, R and Octave – Part 4
b. Deep Learning from first principles in Python, R and Octave – Part 5

6. Elements of Neural Networks and Deep Learning – Part 6
This part discusses initialization methods specifically like He and Xavier. The presentation also focuses on how to prevent over-fitting using regularization. Lastly the dropout method of regularization is also discussed


The corresponding implementations in vectorized R, Python and Octave of the above discussed methods are available in my post Deep Learning from first principles in Python, R and Octave – Part 6

7. Elements of Neural Networks and Deep Learning – Part 7
This presentation introduces exponentially weighted moving average and shows how this is used in different approaches to gradient descent optimization. The key techniques discussed are learning rate decay, momentum method, rmsprop and adam.

The equivalent implementations of the gradient descent optimization techniques in R, Python and Octave can be seen in my post Deep Learning from first principles in Python, R and Octave – Part 7

8. Elements of Neural Networks and Deep Learning – Part 8
This last part touches on the method to adopt while tuning hyper-parameters in Deep Learning networks

Checkout my book ‘Deep Learning from first principles: Second Edition – In vectorized Python, R and Octave’. My book starts with the implementation of a simple 2-layer Neural Network and works its way to a generic L-Layer Deep Learning Network, with all the bells and whistles. The derivations have been discussed in detail. The code has been extensively commented and included in its entirety in the Appendix sections. My book is available on Amazon as paperback ($18.99) and in kindle version($9.99/Rs449).

This concludes this series of presentations on “Elements of Neural Networks and Deep Learning’

Also
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. Introducing cricpy:A python package to analyze performances of cricketers
3. Natural language processing: What would Shakespeare say?
4. Big Data-2: Move into the big league:Graduate from R to SparkR
5. Presentation on Wireless Technologies – Part 1
6. Introducing cricketr! : An R package to analyze performances of cricketers

To see all posts click Index of posts

Pitching yorkpy…swinging away from the leg stump to IPL – Part 3


Clocks offer at best a convenient fiction They imply that time ticks steadily, predictably forward, when our experience shows that it often does the opposite: it stretches and compresses, skips a beat and doubles back.

                                 David Eagleman
                                 

Memory is the space in which a thing happens for a second time

                                 Paul Auster
      

Introduction

In this 3rd post, yorkpy, the python avatar of my R package yorkr develops more muscle. The first two posts of yorkpy were

1. Pitching yorkpy . short of good length to IPL – Part 1 This post dealt with function which perform analytics on an IPL match between any 2 IPL teams
2. Pitching yorkpy…on the middle and outside off-stump to IPL – Part 2 The second post dealt with analytics on all matches between any 2 IPL teams.

This third post deals with analyses and analytics of an IPL team in all matches against all other IPL teams. The data for yorkpy comes from Cricsheet. The data in Cricsheet are in the form of yaml files. These files have already been converted as dataframes and stored as CSV as seen in the earlier posts.You can download all the data used in this post and the previous post at yorkpyData

The signatures of yorkpy and yorkr are identical and will work in almost the same way. However there may be some unique functions in yorkr & yorkpy, based on what my thought process was on that day!

-You can clone/download the code at Github yorkpy
-This post has been published to RPubs at yorkpy-Part3
-Download this post as PDF at IPLT20-yorkpy-part3
-You can download all the data used in this post and the previous post at yorkpyData

The IPL T20 functions in yorkpy are shown below

2. Get data for all T20 matches between an IPL team and all other IPL teams

We can get all IPL T20 matches between an IPL team  and all other teams using the function below. The dir parameter should point to the folder which has the IPL T20 csv files of the individual matches (see Pitching yorkpy…short of good length to IPL-Part 1). This function creates a data frame of all the IPL T20 matches between the IPL team and all other teams and and also saves the dataframe as CSV file if save=True. If save=False the dataframe is just returned and not saved.

import pandas as pd
import os
import yorkpy.analytics as yka
#dir1= "C:\\software\\cricket-package\\yorkpyPkg\\yorkpyData\\IPLConverted"
#getAllMatchesAllOpposition("Kolkata Knight Riders",dir=dir1,save=True)

3. Save data for all matches between an IPL team and all oppositions

This can be done locally using the function below. You could use this function to get combine all IPL matches of an IPL team against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
#dir1= "C:\\software\\cricket-package\\yorkpyPkg\\yorkpyData\\IPLConverted"
#saveAllMatchesAllOppositionIPLT20(dir1)

Note: In the functions below, I have randomly chosen an IPL team for the analyses. You are free to choose any IPL team for your analysis

4.Team Batsmen partnership in Twenty20 (all matches against all IPL teams – summary)

The function below computes the highest partnerships for an IPL team against all other IPL teams for e.g. the batsmen with the highest partnership from Chennai Super Kings in all matches against all other IPL teams. Any other IPL team could have also been chosen.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Chennai Super Kings-allMatchesAllOpposition.csv") 
csk_matches = pd.read_csv(path)
m=yka.teamBatsmenPartnershiAllOppnAllMatches(csk_matches,'Chennai Super Kings',report="summary")
print(m)
##         batsman  totalPartnershipRuns
## 42     SK Raina                  3699
## 28     MS Dhoni                  2986
## 25   MEK Hussey                  1768
## 24      M Vijay                  1600
## 36  S Badrinath                  1441

5. Team Batsmen partnership in Twenty20 (all matches against all IPL teams -detailed)

The function below gives the detailed breakup of partnerships for Mumbai Indian against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Mumbai Indians-allMatchesAllOpposition.csv")
mi_matches = pd.read_csv(path)
theTeam='Mumbai Indians'
m=yka.teamBatsmenPartnershiAllOppnAllMatches(mi_matches,theTeam,report="detailed", top=3)
print(m)
##        batsman  totalPartnershipRuns      non_striker  partnershipRuns
## 0    RG Sharma                3037.0        A Symonds            142.0
## 1    RG Sharma                3037.0      AC Blizzard              5.0
## 2    RG Sharma                3037.0         AJ Finch              2.0
## 3    RG Sharma                3037.0          AP Tare             32.0
## 4    RG Sharma                3037.0        AT Rayudu            566.0
## 5    RG Sharma                3037.0          BR Dunk              1.0
## 6    RG Sharma                3037.0      CJ Anderson            183.0
## 7    RG Sharma                3037.0        CM Gautam             22.0
## 8    RG Sharma                3037.0         DR Smith             50.0
## 9    RG Sharma                3037.0       GJ Maxwell              6.0
## 10   RG Sharma                3037.0         HH Gibbs            109.0
## 11   RG Sharma                3037.0        HH Pandya            105.0
## 12   RG Sharma                3037.0  Harbhajan Singh             86.0
## 13   RG Sharma                3037.0       JC Buttler            105.0
## 14   RG Sharma                3037.0     JEC Franklin             50.0
## 15   RG Sharma                3037.0       KA Pollard            633.0
## 16   RG Sharma                3037.0       KD Karthik            170.0
## 17   RG Sharma                3037.0        KH Pandya             34.0
## 18   RG Sharma                3037.0        KV Sharma             33.0
## 19   RG Sharma                3037.0      LMP Simmons            172.0
## 20   RG Sharma                3037.0       MEK Hussey             21.0
## 21   RG Sharma                3037.0       MJ Guptill             61.0
## 22   RG Sharma                3037.0   MJ McClenaghan              2.0
## 23   RG Sharma                3037.0           N Rana             25.0
## 24   RG Sharma                3037.0         PA Patel            103.0
## 25   RG Sharma                3037.0          RE Levi             25.0
## 26   RG Sharma                3037.0       SL Malinga              0.0
## 27   RG Sharma                3037.0     SR Tendulkar            208.0
## 28   RG Sharma                3037.0        SS Tiwary             27.0
## 29   RG Sharma                3037.0         TL Suman              7.0
## ..         ...                   ...              ...              ...
## 70  KA Pollard                2344.0      CJ Anderson             82.0
## 71  KA Pollard                2344.0        CM Gautam             16.0
## 72  KA Pollard                2344.0         DR Smith             10.0
## 73  KA Pollard                2344.0      DS Kulkarni             15.0
## 74  KA Pollard                2344.0        HH Pandya            158.0
## 75  KA Pollard                2344.0  Harbhajan Singh            158.0
## 76  KA Pollard                2344.0        J Suchith             26.0
## 77  KA Pollard                2344.0       JC Buttler             37.0
## 78  KA Pollard                2344.0     JEC Franklin             38.0
## 79  KA Pollard                2344.0        JP Duminy             63.0
## 80  KA Pollard                2344.0       KD Karthik             40.0
## 81  KA Pollard                2344.0        KH Pandya            111.0
## 82  KA Pollard                2344.0        KV Sharma             13.0
## 83  KA Pollard                2344.0      LMP Simmons             77.0
## 84  KA Pollard                2344.0       MEK Hussey             10.0
## 85  KA Pollard                2344.0       MG Johnson              1.0
## 86  KA Pollard                2344.0           N Rana             60.0
## 87  KA Pollard                2344.0         PA Patel             18.0
## 88  KA Pollard                2344.0          PP Ojha             12.0
## 89  KA Pollard                2344.0         R Dhawan             25.0
## 90  KA Pollard                2344.0        R McLaren             20.0
## 91  KA Pollard                2344.0        R Sathish             27.0
## 92  KA Pollard                2344.0        RG Sharma            587.0
## 93  KA Pollard                2344.0      RJ Peterson              0.0
## 94  KA Pollard                2344.0         S Dhawan             20.0
## 95  KA Pollard                2344.0       SL Malinga             14.0
## 96  KA Pollard                2344.0     SR Tendulkar             69.0
## 97  KA Pollard                2344.0        SS Tiwary             42.0
## 98  KA Pollard                2344.0         TL Suman              2.0
## 99  KA Pollard                2344.0           Z Khan              1.0
## 
## [100 rows x 4 columns]

6. Team Batsmen partnership in Twenty20 – Chart (all matches against all IPL teams)

The function below plots the partnerships of an IPL team against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Delhi Daredevils-allMatchesAllOpposition.csv")
dd_matches = pd.read_csv(path)
yka.teamBatsmenPartnershipAllOppnAllMatchesChart(dd_matches,'Delhi Daredevils', plot=True, top=4, partnershipRuns=100)

7.Team Batsmen partnership in Twenty20 – Dataframe (all matches against all IPL teams)

This function does not plot the data but returns the dataframe to the user to plot or manipulate.

Note: Many of the plots include an additional parameters for e.g. plot which is either True or False. The default value is plot=True. When plot=True the plot will be displayed. When plot=False the data frame will be returned to the user. The user can use this to create an interactive charts. The parameter top= specifies the number of top batsmen that need to be included in the chart, and partnershipRuns gives the minimum cutoff runs in partnwerships to be considered

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kochi Tuskers Kerala-allMatchesAllOpposition.csv")
ktk_matches = pd.read_csv(path)
m=yka.teamBatsmenPartnershipAllOppnAllMatchesChart(ktk_matches,'Kochi Tuskers Kerala', plot=False, top=3, partnershipRuns=100)
print(m)
##              batsman       non_striker  partnershipRuns
## 0        BB McCullum          BJ Hodge             17.0
## 1        BB McCullum  DPMD Jayawardene            160.0
## 2        BB McCullum         M Klinger             67.0
## 3        BB McCullum          PA Patel             40.0
## 4        BB McCullum         RA Jadeja             19.0
## 5        BB McCullum        VVS Laxman             41.0
## 6        BB McCullum  Y Gnaneswara Rao             13.0
## 7   DPMD Jayawardene       BB McCullum            152.0
## 8   DPMD Jayawardene          BJ Hodge             41.0
## 9   DPMD Jayawardene         KM Jadhav              4.0
## 10  DPMD Jayawardene         M Klinger             28.0
## 11  DPMD Jayawardene           OA Shah              9.0
## 12  DPMD Jayawardene          PA Patel             25.0
## 13  DPMD Jayawardene         RA Jadeja             18.0
## 14  DPMD Jayawardene          RV Gomez             10.0
## 15  DPMD Jayawardene        VVS Laxman             12.0
## 16          BJ Hodge       BB McCullum             18.0
## 17          BJ Hodge  DPMD Jayawardene             47.0
## 18          BJ Hodge         KM Jadhav              2.0
## 19          BJ Hodge           OA Shah             19.0
## 20          BJ Hodge          PA Patel             79.0
## 21          BJ Hodge         RA Jadeja             99.0
## 22          BJ Hodge          RV Gomez             21.0

8. Team batsmen versus bowler in Twenty20-Chart (all matches against all IPL teams)

The plots below provide information on how each of the top batsmen of the IPL team fared against the opposition bowlers of all other IPL teams.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Royal Challengers Bangalore-allMatchesAllOpposition.csv")
rcb_matches = pd.read_csv(path)
yka.teamBatsmenVsBowlersAllOppnAllMatches(rcb_matches,"Royal Challengers Bangalore",plot=True,top=3,runsScored=60)

9 Team batsmen versus bowler in Twenty20-Dataframe (all matches against all IPL teams)

This function provides the batting performance of an IPL team against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kings XI Punjab-allMatchesAllOpposition.csv")
kxip_matches = pd.read_csv(path)
m=yka.teamBatsmenVsBowlersAllOppnAllMatches(kxip_matches,'Kings XI Punjab',plot=False,top=2,runsScored=50)
print(m)
##        batsman            bowler  runsScored
## 0     SE Marsh        A Chandila        20.0
## 1     SE Marsh       A Choudhary         1.0
## 2     SE Marsh          A Kumble        37.0
## 3     SE Marsh          A Mishra         0.0
## 4     SE Marsh          A Mithun         9.0
## 5     SE Marsh           A Nehra        33.0
## 6     SE Marsh           A Singh         2.0
## 7     SE Marsh         A Symonds         5.0
## 8     SE Marsh         AA Chavan        19.0
## 9     SE Marsh   AA Jhunjhunwala        15.0
## 10    SE Marsh        AB Agarkar        27.0
## 11    SE Marsh          AB Dinda        31.0
## 12    SE Marsh       AB McDonald         9.0
## 13    SE Marsh         AC Thomas         1.0
## 14    SE Marsh        AD Mathews         7.0
## 15    SE Marsh        AD Russell         8.0
## 16    SE Marsh            AJ Tye         0.0
## 17    SE Marsh        AL Menaria         6.0
## 18    SE Marsh          AM Salvi         8.0
## 19    SE Marsh          AN Ahmed        16.0
## 20    SE Marsh           AS Raut         7.0
## 21    SE Marsh      Ankit Sharma         2.0
## 22    SE Marsh        Ankit Soni        11.0
## 23    SE Marsh           B Kumar        10.0
## 24    SE Marsh             B Lee         1.0
## 25    SE Marsh        BAW Mendis        11.0
## 26    SE Marsh           BB Sran         3.0
## 27    SE Marsh          BJ Hodge        18.0
## 28    SE Marsh      Basil Thampi        17.0
## 29    SE Marsh   C de Grandhomme         8.0
## ..         ...               ...         ...
## 235  DA Miller          R Sharma         7.0
## 236  DA Miller         R Tewatia         3.0
## 237  DA Miller     R Vinay Kumar        30.0
## 238  DA Miller         RA Jadeja        84.0
## 239  DA Miller         RD Chahar         3.0
## 240  DA Miller  RE van der Merwe         5.0
## 241  DA Miller  RN ten Doeschate         1.0
## 242  DA Miller          RP Singh        35.0
## 243  DA Miller       Rashid Khan         0.0
## 244  DA Miller         S Aravind         7.0
## 245  DA Miller            S Kaul        23.0
## 246  DA Miller         S Kaushik         8.0
## 247  DA Miller           S Ladda         6.0
## 248  DA Miller          S Nadeem        11.0
## 249  DA Miller          SK Raina         2.0
## 250  DA Miller        SL Malinga         9.0
## 251  DA Miller   SMSM Senanayake         6.0
## 252  DA Miller         SP Narine        10.0
## 253  DA Miller         SR Watson        16.0
## 254  DA Miller         STR Binny        14.0
## 255  DA Miller   Shakib Al Hasan         3.0
## 256  DA Miller          TA Boult        20.0
## 257  DA Miller        TG Southee        11.0
## 258  DA Miller          UT Yadav        51.0
## 259  DA Miller          VR Aaron        19.0
## 260  DA Miller          VS Malik         3.0
## 261  DA Miller         YK Pathan         0.0
## 262  DA Miller         YS Chahal        35.0
## 263  DA Miller      Yuvraj Singh        11.0
## 264  DA Miller            Z Khan         2.0
## 
## [265 rows x 3 columns]

10. Team batting scorecard(all matches against all IPL teams)

This function provides the overall scorecard for an IPL team in all matches against all other IPL teams. The batting scorecard shows the top batsmen for Kolkata Knight Riders below

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kolkata Knight Riders-allMatchesAllOpposition.csv")
kkr_matches = pd.read_csv(path)
scorecard=yka.teamBattingScorecardAllOppnAllMatches(kkr_matches,'Kolkata Knight Riders')
print(scorecard)
##              batsman    runs  balls   4s  6s          SR
## 19         G Gambhir  3035.0   2533  352  46  119.818397
## 17         YK Pathan  1893.0   1421  150  86  133.216045
## 22        RV Uthappa  1806.0   1311  200  54  137.757437
## 16         JH Kallis  1295.0   1237  128  23  104.688763
## 23         MK Pandey  1270.0   1048  103  38  121.183206
## 0         SC Ganguly  1031.0    977  105  36  105.527124
## 12         MK Tiwary  1002.0    921   86  23  108.794788
## 1        BB McCullum   882.0    754   92  32  116.976127
## 25          SA Yadav   608.0    474   54  21  128.270042
## 15          MS Bisla   543.0    518   60  16  104.826255
## 26        AD Russell   516.0    308   45  34  167.532468
## 4          DJ Hussey   511.0    417   31  28  122.541966
## 24   Shakib Al Hasan   498.0    399   44  15  124.812030
## 10          BJ Hodge   476.0    430   47  10  110.697674
## 11          CH Gayle   463.0    350   45  26  132.285714
## 18        EJG Morgan   444.0    373   45  16  119.034853
## 54           CA Lynn   378.0    250   30  23  151.200000
## 6          LR Shukla   374.0    320   31  15  116.875000
## 29  RN ten Doeschate   326.0    238   26  15  136.974790
## 21            DB Das   304.0    267   23  16  113.857678
## 3            WP Saha   298.0    213   24  12  139.906103
## 28         SP Narine   271.0    193   36  12  140.414508
## 13        AD Mathews   249.0    211   20   8  118.009479
## 33       Salman Butt   193.0    172   30   2  112.209302
## 41        MN van Wyk   167.0    135   19   1  123.703704
## 7         AB Agarkar   160.0    137   12   5  116.788321
## 20          R Bhatia   159.0    134   15   3  118.656716
## 51   C de Grandhomme   126.0     92   10   6  136.956522
## 39         CA Pujara   122.0    119   14   3  102.521008
## 40           OA Shah   115.0     96    7   5  119.791667
## ..               ...     ...    ...  ...  ..         ...
## 50         JO Holder    22.0     20    2   1  110.000000
## 65     Kuldeep Yadav    20.0     22    2   0   90.909091
## 71         BJ Haddin    18.0     11    2   1  163.636364
## 70   NM Coulter-Nile    14.0     13    0   2  107.692308
## 47          L Balaji    13.0     12    1   0  108.333333
## 55   SMSM Senanayake    10.0     17    0   0   58.823529
## 53          M Morkel     9.0      8    0   0  112.500000
## 62          AN Ghosh     7.0      8    1   0   87.500000
## 32           GB Hogg     7.0      6    0   0  116.666667
## 56        MV Boucher     6.0      6    0   0  100.000000
## 77     Azhar Mahmood     6.0      8    1   0   75.000000
## 78          DM Bravo     6.0      5    1   0  120.000000
## 68         SS Shaikh     6.0      7    1   0   85.714286
## 66          TA Boult     5.0      8    0   0   62.500000
## 76    Mohammed Shami     5.0     10    0   0   50.000000
## 80           P Dogra     5.0      8    0   0   62.500000
## 69     R Vinay Kumar     4.0      7    0   0   57.142857
## 75        AS Rajpoot     4.0      7    1   0   57.142857
## 43     Mandeep Singh     4.0     11    1   0   36.363636
## 37          AB Dinda     4.0      8    0   0   50.000000
## 79        PJ Sangwan     4.0      2    1   0  200.000000
## 73         R McLaren     3.0      6    0   0   50.000000
## 67         SB Bangar     2.0      9    0   0   22.222222
## 57       RS Gavaskar     2.0      8    0   0   25.000000
## 72     Shoaib Akhtar     2.0      8    0   0   25.000000
## 38  Mashrafe Mortaza     2.0      2    0   0  100.000000
## 63        BAW Mendis     1.0      2    0   0   50.000000
## 58           SE Bond     1.0      2    0   0   50.000000
## 44     CK Langeveldt     0.0      1    0   0    0.000000
## 30        PJ Cummins     0.0      2    0   0    0.000000
## 
## [81 rows x 6 columns]

10a. Team batting scorecard(all matches against all IPL teams)

The output below shows the Chennai Super Kings against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Chennai Super Kings-allMatchesAllOpposition.csv")
csk_matches = pd.read_csv(path)
scorecard=yka.teamBattingScorecardAllOppnAllMatches(csk_matches,'Chennai Super Kings')
print(scorecard)
##             batsman  runs  balls   4s   6s          SR
## 3          SK Raina  3699   2735  322  150  135.246801
## 5          MS Dhoni  2986   2199  218  126  135.788995
## 17       MEK Hussey  1768   1461  181   45  121.013005
## 11          M Vijay  1600   1289  141   66  124.127230
## 4       S Badrinath  1441   1245  154   28  115.742972
## 9         ML Hayden  1107    838  121   44  132.100239
## 18     F du Plessis  1081    867   92   29  124.682814
## 25         DR Smith   965    766  102   50  125.979112
## 26      BB McCullum   841    634   83   42  132.649842
## 6         JA Morkel   827    591   51   48  139.932318
## 20         DJ Bravo   706    543   54   30  130.018416
## 19        RA Jadeja   670    533   46   23  125.703565
## 0          PA Patel   516    529   67    7   97.542533
## 2        SP Fleming   196    171   27    3  114.619883
## 13         R Ashwin   190    208   19    1   91.346154
## 21         S Vidyut   145    115   21    3  126.086957
## 31          WP Saha   144    138    8    8  104.347826
## 1        S Anirudha   133    116    9    7  114.655172
## 33        DJ Hussey   116     96    8    6  120.833333
## 38           P Negi   116     77   10    5  150.649351
## 10         JDP Oram   106    107    6    5   99.065421
## 29        GJ Bailey    63     67    9    0   94.029851
## 22       A Flintoff    62     57    5    2  108.771930
## 8           MS Gony    50     39    2    5  128.205128
## 7   Joginder Sharma    36     30    1    2  120.000000
## 27         M Manhas    35     26    3    1  134.615385
## 28        MM Sharma    29     26    1    2  111.538462
## 23        SB Jakati    27     28    3    0   96.428571
## 12          JM Kemp    26     25    1    1  104.000000
## 14         L Balaji    22     35    1    1   62.857143
## 24     DE Bollinger    21     23    1    1   91.304348
## 41    CK Kapugedera    16     24    0    0   66.666667
## 37        CH Morris    14     17    0    0   82.352941
## 30       T Thushara    12     19    0    0   63.157895
## 42          M Ntini    11     19    2    0   57.894737
## 15   M Muralitharan     9     13    1    0   69.230769
## 32  KMDN Kulasekara     5      3    1    0  166.666667
## 34        SB Styris     5      2    1    0  250.000000
## 35       B Laughlin     4      9    0    0   44.444444
## 16          S Tyagi     3      4    0    0   75.000000
## 45  KB Arun Karthik     3      5    0    0   60.000000
## 36       AS Rajpoot     2      6    0    0   33.333333
## 43          RG More     2      2    0    0  100.000000
## 44         S Randiv     2      4    0    0   50.000000
## 39          A Nehra     1      7    0    0   14.285714
## 40         A Mukund     0      1    0    0    0.000000

11.Team Bowling scorecard (all matches against all IPL teams)

The output below gives the bowling performance of an IPL team against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Sunrisers Hyderabad-allMatchesAllOpposition.csv")
srh_matches = pd.read_csv(path)
scorecard=yka.teamBowlingScorecardAllOppnAllMatches(srh_matches,'Sunrisers Hyderabad')
## C:\Users\Ganesh\ANACON~1\lib\site-packages\yorkpy\analytics.py:564: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
##   df1['over']=df1.delivery.astype(int)
## C:\Users\Ganesh\ANACON~1\lib\site-packages\yorkpy\analytics.py:567: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
##   df1['runsConceded']=df1['runs'] + df1['wides'] + df1['noballs']
print(scorecard)
##               bowler  overs  runs  maidens  wicket   econrate
## 60       JP Faulkner     28   192        0      15   6.857143
## 83         MM Sharma     37   334        0      13   9.027027
## 119       SL Malinga     31   215        0      13   6.935484
## 123        SR Watson     30   281        0      13   9.366667
## 90   NM Coulter-Nile     24   166        0      12   6.916667
## 31          DJ Bravo     26   184        0      12   7.076923
## 135         UT Yadav     37   297        0      12   8.027027
## 125   Sandeep Sharma     32   280        0      11   8.750000
## 75          M Morkel     25   195        0       9   7.800000
## 81    MJ McClenaghan     24   175        0       9   7.291667
## 5           AB Dinda     23   165        0       9   7.173913
## 55        JD Unadkat     20   167        0       8   8.350000
## 36       DS Kulkarni     28   200        0       8   7.142857
## 25         CH Morris     24   190        0       7   7.916667
## 101         R Bhatia     18   128        0       7   7.111111
## 70     Kuldeep Yadav     16   129        0       7   8.062500
## 11          AR Patel     27   208        0       7   7.703704
## 122        SP Narine     43   282        0       7   6.558140
## 141        YS Chahal     26   224        0       6   8.615385
## 44   Harbhajan Singh     39   264        0       6   6.769231
## 96         PP Chawla     21   140        0       6   6.666667
## 4            A Zampa      4    19        0       6   4.750000
## 126  Shakib Al Hasan     14    99        1       6   7.071429
## 80        MG Johnson     20   155        0       6   7.750000
## 59         JP Duminy     10    80        0       5   8.000000
## 58         JO Holder     15   113        0       5   7.533333
## 92           P Kumar     23   173        0       5   7.521739
## 100         R Ashwin     28   142        0       5   5.071429
## 2           A Mishra     18   144        0       4   8.000000
## 106    R Vinay Kumar     19   154        0       4   8.105263
## ..               ...    ...   ...      ...     ...        ...
## 6     AD Mascarenhas      4    25        0       0   6.250000
## 13        Ankit Soni      2    31        0       0  15.500000
## 132          TM Head      1    11        0       0  11.000000
## 10          AN Ahmed      6    63        0       0  10.500000
## 131       TM Dilshan      1    10        0       0  10.000000
## 134     Tejas Baroka      3    33        0       0  11.000000
## 73          M Ashwin      1     6        0       0   6.000000
## 109        RG Sharma      1     5        0       0   5.000000
## 22      Basil Thampi      2    21        0       0  10.500000
## 23           C Munro      1     8        0       0   8.000000
## 68         KV Sharma      2    19        0       0   9.500000
## 77           M Vijay      4    24        0       0   6.000000
## 66         KJ Abbott      3    34        0       0  11.333333
## 65         KH Pandya      2    17        0       0   8.500000
## 82          MM Patel      3    22        0       0   7.333333
## 62          K Rabada      4    59        0       0  14.750000
## 85        MP Stoinis      3    28        0       0   9.333333
## 54         JA Morkel      3    35        0       0  11.666667
## 46          I Sharma      8    64        0       0   8.000000
## 94        PJ Cummins      4    37        0       0   9.250000
## 95        PJ Sangwan      8    82        0       0  10.250000
## 103        R Sathish      1     9        0       0   9.000000
## 38          DW Steyn      2    17        0       0   8.500000
## 108          RG More      2    28        0       0  14.000000
## 34         DJG Sammy      2    18        0       0   9.000000
## 33     DJ Muthuswami      2    20        0       0  10.000000
## 32          DJ Hooda      5    45        0       0   9.000000
## 24          CH Gayle      3    24        0       0   8.000000
## 116        SA Abbott      2    21        0       0  10.500000
## 72         LR Shukla      2    28        0       0  14.000000
## 
## [144 rows x 6 columns]

11a.Team Bowling scorecard (all matches against all IPL teams)

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Rajasthan Royals-allMatchesAllOpposition.csv")
rr_matches = pd.read_csv(path)
scorecard=yka.teamBowlingScorecardAllOppnAllMatches(rr_matches,'Rajasthan Royals')
print(scorecard)
##                bowler  overs  runs  maidens  wicket   econrate
## 2            A Mishra     63   426        0      29   6.761905
## 66          JA Morkel     38   301        0      16   7.921053
## 129     R Vinay Kumar     48   406        0      15   8.458333
## 135          RP Singh     41   255        0      14   6.219512
## 95        MF Maharoof     23   139        0      14   6.043478
## 118         PP Chawla     45   353        0      14   7.844444
## 130         RA Jadeja     32   227        0      14   7.093750
## 50           DW Steyn     43   232        0      13   5.395349
## 56    Harbhajan Singh     45   341        0      12   7.577778
## 1            A Kumble     21   108        1      12   5.142857
## 159        SL Malinga     49   363        0      12   7.408163
## 60          IK Pathan     37   279        0      11   7.540541
## 82         KA Pollard     21   201        0      11   9.571429
## 119           PP Ojha     46   426        0      11   9.260870
## 121          R Ashwin     29   222        0      11   7.655172
## 22            B Kumar     31   233        0      11   7.516129
## 3             A Nehra     32   214        0      11   6.687500
## 41           DJ Bravo     30   292        0      10   9.733333
## 110           P Kumar     48   329        1      10   6.854167
## 58           I Sharma     37   284        0       9   7.675676
## 168   Shakib Al Hasan     25   153        0       9   6.120000
## 87           L Balaji     33   277        0       9   8.393939
## 122          R Bhatia     19   121        0       8   6.368421
## 48        DS Kulkarni     21   148        0       8   7.047619
## 101         MM Sharma     20   142        0       8   7.100000
## 174          UT Yadav     25   203        0       8   8.120000
## 15           AR Patel     16   110        0       7   6.875000
## 133         RJ Harris     16   132        0       7   8.250000
## 72          JH Kallis     37   254        0       7   6.864865
## 192            Z Khan     33   213        0       7   6.454545
## ..                ...    ...   ...      ...     ...        ...
## 170      Shoaib Ahmed      2    19        0       0   9.500000
## 54          GS Sandhu      4    49        0       0  12.250000
## 139          RV Gomez      1     9        0       0   9.000000
## 163         SPD Smith      0     5        0       0        inf
## 115       PC Valthaty      3    35        0       0  11.666667
## 34        CJ Anderson      4    26        0       0   6.500000
## 81         K Upadhyay      3    29        0       0   9.666667
## 79             K Goel      1    11        0       0  11.000000
## 28          BJ Rohrer      1    12        0       0  12.000000
## 78    Joginder Sharma      2    23        0       0  11.500000
## 99          MK Tiwary      2    28        0       0  14.000000
## 26       BE Hendricks      4    57        0       0  14.250000
## 102          MR Marsh      1    10        0       0  10.000000
## 106       NL McCullum      3    22        0       0   7.333333
## 113        P Prasanth      1    18        0       0  18.000000
## 114           P Suyal      4    45        0       0  11.250000
## 46      DP Vijaykumar      1    10        0       0  10.000000
## 154         SB Styris      2    14        0       0   7.000000
## 71       JEC Franklin      3    32        0       0  10.666667
## 70          JE Taylor      3    22        0       0   7.333333
## 18       Ankit Sharma      4    33        0       0   8.250000
## 134  RN ten Doeschate      2    14        0       0   7.000000
## 16       Abdur Razzak      2    29        0       0  14.500000
## 65           J Theron      6    48        0       0   8.000000
## 146          S Narwal      2    17        0       0   8.500000
## 63            J Botha      1    19        0       0  19.000000
## 149           S Tyagi      8    65        0       0   8.125000
## 151         SB Bangar      2    20        0       0  10.000000
## 13           AM Nayar      2     7        0       0   3.500000
## 0      A Ashish Reddy      3    22        0       0   7.333333
## 
## [193 rows x 6 columns]

12. Team Bowling wicket kind -Chart (all matches against all IPL teams)

The functions compute and display the kind of wickets taken(bowled, caught, lbw etc) by an IPL team in all matches against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Gujarat Lions-allMatchesAllOpposition.csv")
gl_matches = pd.read_csv(path)
yka.teamBowlingWicketKindAllOppnAllMatches(gl_matches,'Gujarat Lions',plot=True,top=5,wickets=2)

13. Team Bowling wicket kind -Dataframe (all matches against all IPL teams)

This gives the type of wickets taken for an IPL team against all other IPL teams.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Rising Pune Supergiants-allMatchesAllOpposition.csv")
rps_matches = pd.read_csv(path)
m=yka.teamBowlingWicketKindAllOppnAllMatches(rps_matches,'Rising Pune Supergiants',plot=False,top=4,wickets=10)
print(m)
##           bowler               kind  wickets
## 0        A Nehra             caught        4
## 1        A Nehra            run out        2
## 2      MM Sharma             caught        3
## 3      MM Sharma  caught and bowled        1
## 4      MM Sharma            run out        1
## 5      SR Watson             bowled        1
## 6      SR Watson             caught        4
## 7  KW Richardson             caught        3
## 8  KW Richardson       retired hurt        1

14 Team Bowler vs Batman -Plot (all matches against all IPL teams)

The function below gives the performance of bowlers against batsmen ,in all matches against another IPL team.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Rising Pune Supergiants-allMatchesAllOpposition.csv")
rps_matches = pd.read_csv(path)
yka.teamBowlersVsBatsmenAllOppnAllMatches(rps_matches,'Rising Pune Supergiants',plot=True,top=5,runsConceded=10)

15 Team Bowler vs Batman – Dataframe (all matches against all IPL teams)

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Deccan Chargers-allMatchesAllOpposition.csv")
dc_matches = pd.read_csv(path)
m=yka.teamBowlersVsBatsmenAllOppnAllMatches(dc_matches,'Deccan Chargers',plot=False,top=2,runsConceded=30)
print(m)
##        bowler          batsman  runsConceded
## 0     P Kumar   A Ashish Reddy           6.0
## 1     P Kumar        A Symonds          15.0
## 2     P Kumar      AA Bilakhia          12.0
## 3     P Kumar  AA Jhunjhunwala           1.0
## 4     P Kumar     AC Gilchrist          20.0
## 5     P Kumar    Anirudh Singh          11.0
## 6     P Kumar         B Chipli           1.0
## 7     P Kumar         CL White          11.0
## 8     P Kumar     DB Ravi Teja          15.0
## 9     P Kumar        DJ Harris           2.0
## 10    P Kumar         DR Smith           5.0
## 11    P Kumar       FH Edwards           3.0
## 12    P Kumar         HH Gibbs          46.0
## 13    P Kumar         J Theron           0.0
## 14    P Kumar        JP Duminy           4.0
## 15    P Kumar    KC Sangakkara          15.0
## 16    P Kumar        MD Mishra           4.0
## 17    P Kumar         PA Patel           9.0
## 18    P Kumar        RG Sharma          36.0
## 19    P Kumar        RJ Harris           3.0
## 20    P Kumar         S Dhawan          37.0
## 21    P Kumar          S Sohal           6.0
## 22    P Kumar        SB Styris           6.0
## 23    P Kumar    Shahid Afridi           0.0
## 24    P Kumar         TL Suman          22.0
## 25    P Kumar       VVS Laxman           5.0
## 26    P Kumar  Y Venugopal Rao           1.0
## 27  PP Chawla   A Ashish Reddy           2.0
## 28  PP Chawla        A Symonds          35.0
## 29  PP Chawla  AA Jhunjhunwala           6.0
## 30  PP Chawla     AC Gilchrist           4.0
## 31  PP Chawla         B Chipli           8.0
## 32  PP Chawla         CL White          16.0
## 33  PP Chawla     DB Ravi Teja          30.0
## 34  PP Chawla        DJ Harris           9.0
## 35  PP Chawla        DNT Zoysa           1.0
## 36  PP Chawla         HH Gibbs          30.0
## 37  PP Chawla        JP Duminy          10.0
## 38  PP Chawla    KC Sangakkara           1.0
## 39  PP Chawla         MR Marsh           1.0
## 40  PP Chawla         PA Patel           4.0
## 41  PP Chawla         PA Reddy           8.0
## 42  PP Chawla        RG Sharma          50.0
## 43  PP Chawla         S Dhawan          33.0
## 44  PP Chawla        SB Bangar           1.0
## 45  PP Chawla         TL Suman          17.0
## 46  PP Chawla       VVS Laxman           7.0
## 47  PP Chawla  Y Venugopal Rao           3.0

16 Team Wins and Losses – Summary (all matches against all IPL teams)

The function below computes and plots the number of wins and losses between an IPL team and all other IPL teams in all matches. The summary just gives the wins, losses and ties

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Chennai Super Kings-allMatchesAllOpposition.csv")
csk_matches = pd.read_csv(path)
team1='Chennai Super Kings'
yka.plotWinLossByTeamAllOpposition(csk_matches,team1,plot="summary")

16a Team Wins and Losses – Detailed (all matches against all IPL teams)

The function below computes and plot the number of wins and losses between an IPL team and all other IPL teams in all matches. This gives a breakup of which team won against this team.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Chennai Super Kings-allMatchesAllOpposition.csv")
csk_matches = pd.read_csv(path)
team1='Chennai Super Kings'
yka.plotWinLossByTeamAllOpposition(csk_matches,team1,plot="detailed")

16b Team Wins and Losses – Summary (all matches against all IPL teams)

This plot gives the wins vs losses of MI against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Mumbai Indians-allMatchesAllOpposition.csv")
mi_matches = pd.read_csv(path)
team1='Mumbai Indians'
yka.plotWinLossByTeamAllOpposition(mi_matches,team1,plot="summary")

16c Team Wins and Losses – Detailed (all matches against all IPL teams)

The function below computes and plot the number of wins and losses between an IPL team and all other IPL teams in all matches. This gives the breakup of MI wins, losses and ties

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Mumbai Indians-allMatchesAllOpposition.csv")
mi_matches = pd.read_csv(path)
team1='Mumbai Indians'
yka.plotWinLossByTeamAllOpposition(mi_matches,team1,plot="detailed")

17 Team Wins by win type (all matches against all IPL teams)

This function shows how the win happened whether by runs or by wickets in all matches played against all other IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Royal Challengers Bangalore-allMatchesAllOpposition.csv")
rcb_matches = pd.read_csv(path)
yka.plotWinsByRunOrWicketsAllOpposition(rcb_matches,'Royal Challengers Bangalore')

18 Team Wins by toss decision (summary) (all matches against all IPL teams)

This show how Royal Challengers Bangalore fared when it chose to field on winning the toss

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Royal Challengers Bangalore-allMatchesAllOpposition.csv")
rcb_matches = pd.read_csv(path)
yka.plotWinsbyTossDecisionAllOpposition(rcb_matches,'Royal Challengers Bangalore',tossDecision='field',plot='summary')

18a. Team Wins by toss decision (detailed) (all matches against all IPL teams)

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kings XI Punjab-allMatchesAllOpposition.csv")
kxip_matches = pd.read_csv(path)
yka.plotWinsbyTossDecisionAllOpposition(kxip_matches,'Kings XI Punjab',tossDecision='field',plot='detailed')

19 Team Wins by toss decision (summary) (all matches against all IPL teams)

This plot shows how Mumbai Indians fared when it chose to bat on winning the toss against all other IPL teams.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Delhi Daredevils-allMatchesAllOpposition.csv")
mi_rcb_matches = pd.read_csv(path)
yka.plotWinsbyTossDecisionAllOpposition(mi_rcb_matches,'Mumbai Indians',tossDecision='bat',plot='summary')

20 Team Wins by toss decision (detailed)(all matches against all IPL teams)

This plot shows how Kings X1 Punjab fared when it chose to bat on winning the toss

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data2"
path=os.path.join(dir1,"Kings XI Punjab-allMatchesAllOpposition.csv")
kxip_matches = pd.read_csv(path)
yka.plotWinsbyTossDecisionAllOpposition(kxip_matches,'Kings XI Punjab',tossDecision='bat',plot='detailed')

Feel free to clone/download the code from Github yorkpy

Conclusion

This post included analysis of an IPL team against all other IPL teams. You can download the data for this and the earlier posts from [yorkpyData](https://github.com/tvganesh/yorkpyData

The code can be cloned/downloaded from Github

To be continued. Watch this space!

Also see
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. My book ‘Deep Learning from first principles:Second Edition’ now on Amazon
3. Designing a Social Web Portal
4. Computer Vision: Ramblings on derivatives, histograms and contours
5. Introducing cricket package yorkr: Part 3-Foxed by flight!
6. The making of Total Control Android game

To see all posts click Index of posts

Pitching yorkpy…on the middle and outside off-stump to IPL – Part 2


When you come to a fork in the road, take it.
You’ve got to be very careful if you don’t know where you are going, because you might not get there

      Yogi Berra

Try taking his (Rahul Dravid’s) wicket in the first 15 minutes. If you can’t then only try to take the remaining wickets

      Steve Waugh
      

Introduction

This post is a follow-up to my previous post, Pitching yorkpy…short of good length to IPL-Part 1, in which I analyzed individual IPL matches. In this 2nd post I analyze the data in all matches between any 2 IPL teams, say CSK-RCB, MI-KKR or DD-RPS and so on. As I have already mentioned yorky is the python clone of my R packkage yorkr and this post is almost a mirror image of my post with yorkr namely yorkr crashes the IPL party! – Part 2. The signatures of yorkpy and yorkr are identical and will work in amost the same way. yorkpy, like yorkr, uses data from Cricsheet

You can clone/download the code at Github yorkpy
This post has been published to RPubs at yorkpy-Part2
You can download this post as PDF at IPLT20-yorkpy-part2
You can download all the data used in this post and the previous post at yorkpyData

2. Get data for all T20 matches between 2 teams

We can get all IPL T20 matches between any 2 teams using the function below. The dir parameter should point to the folder which has the IPL T20 csv files of the individual matches (see Pitching yorkpy…short of good length to IPL-Part 1). This function creates a data frame of all the IPL T20 matches and and also saves the dataframe as CSV file if save=True. If save=False the dataframe is just returned and not saved.

import pandas as pd
import os
import yorkpy.analytics as yka
#dir1= "C:\\software\\cricket-package\\yorkpyPkg\\yorkpyData\\IPLConverted"
#yka.getAllMatchesBetweenTeams("Kolkata Knight Riders","Delhi Daredevils",dir=dir1,save=True)

3. Save data for all matches between all combination of 2 teams

This can be done locally using the function below. You could use this function to combine all IPL Twenty20 matches between any 2 IPL teams into a single dataframe and save it in the current folder. All the dataframes for all combinations have already been done and are available as CSV files in Github at yorkpyData

import pandas as pd
import os
import yorkpy.analytics as yka
#dir1= "C:\\software\\cricket-package\\yorkpyPkg\\yorkpyData\\IPLConverted"
#yka.saveAllMatchesBetween2IPLTeams(dir1)

Note: In the functions below, I have randomly chosen any 2 IPL teams and analyze how the teams have performed against each other in different areas. You are free to choose any 2 combination of IPL teams for your analysis

4.Team Batsmen partnership in Twenty20 (all matches with opposing IPL team – summary)

The function below computes the highest partnerships between the 2 IPL teams Chennai Superkings and Delhi Daredevils. Any other 2 IPL team could have also been chosen. The summary gives the top 3 batsmen for Delhi Daredevils namely Sehwag, Gambhir and Dinesh Karthik when the report=‘summary’

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Chennai Super Kings-Delhi Daredevils-allMatches.csv")
csk_dd_matches = pd.read_csv(path)
m=yka.teamBatsmenPartnershiOppnAllMatches(csk_dd_matches,'Delhi Daredevils',report="summary")
print(m)
##            batsman  totalPartnershipRuns
## 49        V Sehwag                   233
## 12       G Gambhir                   200
## 21      KD Karthik                   180
## 10       DA Warner                   134
## 4   AB de Villiers                   133

5. Team Batsmen partnership in Twenty20 (all matches with opposing IPL team -detailed)

The function below gives the detailed breakup of partnerships between Deccan Chargers and Mumbai Indians for Deccan Chargers.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Deccan Chargers-Mumbai Indians-allMatches.csv")
dc_mi_matches = pd.read_csv(path)
theTeam='Deccan Chargers'
m=yka.teamBatsmenPartnershiOppnAllMatches(dc_mi_matches,theTeam,report="detailed", top=4)
print(m)
##          batsman  totalPartnershipRuns      non_striker  partnershipRuns
## 0   AC Gilchrist                   201        A Symonds                0
## 1   AC Gilchrist                   201         HH Gibbs               53
## 2   AC Gilchrist                   201        MD Mishra                0
## 3   AC Gilchrist                   201        RG Sharma               20
## 4   AC Gilchrist                   201    Shahid Afridi                6
## 5   AC Gilchrist                   201         TL Suman                7
## 6   AC Gilchrist                   201       VVS Laxman              115
## 7       S Dhawan                   122         A Mishra                9
## 8       S Dhawan                   122         B Chipli                1
## 9       S Dhawan                   122         CL White                2
## 10      S Dhawan                   122     DT Christian               52
## 11      S Dhawan                   122         IR Jaggi                2
## 12      S Dhawan                   122        JP Duminy                9
## 13      S Dhawan                   122    KC Sangakkara               16
## 14      S Dhawan                   122         PA Patel               22
## 15      S Dhawan                   122          S Sohal                9
## 16     RG Sharma                   103        A Symonds               11
## 17     RG Sharma                   103     AC Gilchrist               18
## 18     RG Sharma                   103         DR Smith                6
## 19     RG Sharma                   103         HH Gibbs                3
## 20     RG Sharma                   103   Jaskaran Singh               15
## 21     RG Sharma                   103        KAJ Roach                4
## 22     RG Sharma                   103        LPC Silva                0
## 23     RG Sharma                   103         TL Suman               14
## 24     RG Sharma                   103  Y Venugopal Rao               32
## 25      HH Gibbs                   102     AC Gilchrist               40
## 26      HH Gibbs                   102         DR Smith               24
## 27      HH Gibbs                   102        MD Mishra               27
## 28      HH Gibbs                   102        RG Sharma                8
## 29      HH Gibbs                   102       VVS Laxman                1
## 30      HH Gibbs                   102  Y Venugopal Rao                2

6. Team Batsmen partnership in Twenty20 – Chart (all matches with opposing IPL team)

The function below plots the partnerships in all matches between 2 IPL teams and plots as chart

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Gujarat Lions-Kings XI Punjab-allMatches.csv")
gl_kxip_matches = pd.read_csv(path)
yka.teamBatsmenPartnershipOppnAllMatchesChart(gl_kxip_matches,'Kings XI Punjab','Gujarat Lions', plot=True, top=4, partnershipRuns=20)

7.Team Batsmen partnership in Twenty20 – Dataframe (all matches with opposing IPL team)

This function does not plot the data but returns the dataframe to the user to plot or manipulate.

Note: Many of the plots include an additional parameters for e.g. plot which is either True or False. The default value is plot=True. When plot=True the plot will be displayed. When plot=False the data frame will be returned to the user. The user can use this to create an interactive charts. The parameter top= specifies the number of top batsmen that need to be included in the chart, and partnershipRuns gives the minimum cutoff runs in partnerships to be considered

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Kolkata Knight Riders-Rising Pune Supergiants-allMatches.csv")
kkr_rps_matches = pd.read_csv(path)
m=yka.teamBatsmenPartnershipOppnAllMatchesChart(kkr_rps_matches,'Rising Pune Supergiants','Kolkata Knight Riders', plot=False, top=5, partnershipRuns=20)
print(m)
##         batsman   non_striker  partnershipRuns
## 0     AM Rahane  F du Plessis               20
## 1     AM Rahane     JA Morkel               16
## 2     AM Rahane   NLTC Perera                6
## 3     AM Rahane     SPD Smith               25
## 4     AM Rahane    UT Khawaja                2
## 5     GJ Bailey     IK Pathan                4
## 6     GJ Bailey     SS Tiwary               28
## 7     GJ Bailey    UT Khawaja                1
## 8      MS Dhoni     IK Pathan                5
## 9      MS Dhoni     JA Morkel                1
## 10     MS Dhoni   NLTC Perera                2
## 11     MS Dhoni      R Ashwin                1
## 12     MS Dhoni      R Bhatia               22
## 13    SPD Smith     AM Rahane               31
## 14  NLTC Perera     AM Rahane               12
## 15  NLTC Perera      MS Dhoni               13

8. Team batsmen versus bowler in Twenty20-Chart (all matches with opposing IPL team)

The plots below provide information on how each of the top batsmen of the IPL teams fared against the opposition bowlers

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Rajasthan Royals-Royal Challengers Bangalore-allMatches.csv")
rr_rcb_matches = pd.read_csv(path)
yka.teamBatsmenVsBowlersOppnAllMatches(rr_rcb_matches,'Rajasthan Royals',"Royal Challengers Bangalore",plot=True,top=3,runsScored=20)

9 Team batsmen versus bowler in Twenty20-Dataframe (all matches with opposing IPL team)

This function provides the bowling performance, the number of overs bowled, maidens, runs conceded. wickets taken and economy rate for the IPL match

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Mumbai Indians-Delhi Daredevils-allMatches.csv")
mi_dd_matches = pd.read_csv(path)
m=yka.teamBatsmenVsBowlersOppnAllMatches(mi_dd_matches,'Delhi Daredevils',"Mumbai Indians",plot=False,top=2,runsScored=50)
print(m)
##       batsman           bowler  runsScored
## 0    V Sehwag          A Nehra         6.0
## 1    V Sehwag       AG Murtaza         6.0
## 2    V Sehwag         AM Nayar        14.0
## 3    V Sehwag         CJ McKay        10.0
## 4    V Sehwag     CRD Fernando         9.0
## 5    V Sehwag         DJ Bravo         9.0
## 6    V Sehwag      DJ Thornely         0.0
## 7    V Sehwag         DR Smith        13.0
## 8    V Sehwag      DS Kulkarni        20.0
## 9    V Sehwag  Harbhajan Singh        54.0
## 10   V Sehwag        JJ Bumrah        19.0
## 11   V Sehwag       KA Pollard        37.0
## 12   V Sehwag         MM Patel        27.0
## 13   V Sehwag          PP Ojha         7.0
## 14   V Sehwag         R Shukla         9.0
## 15   V Sehwag      RJ Peterson         7.0
## 16   V Sehwag         RP Singh        28.0
## 17   V Sehwag       SL Malinga        32.0
## 18   V Sehwag       SM Pollock        25.0
## 19   V Sehwag    ST Jayasuriya        29.0
## 20   V Sehwag           Z Khan        14.0
## 21  JP Duminy      CJ Anderson         3.0
## 22  JP Duminy        HH Pandya         7.0
## 23  JP Duminy  Harbhajan Singh        29.0
## 24  JP Duminy        J Suchith         5.0
## 25  JP Duminy        JJ Bumrah        70.0
## 26  JP Duminy       KA Pollard        29.0
## 27  JP Duminy        KH Pandya         8.0
## 28  JP Duminy       M de Lange         6.0
## 29  JP Duminy   MJ McClenaghan        14.0
## 30  JP Duminy           N Rana         1.0
## 31  JP Duminy          PP Ojha        16.0
## 32  JP Duminy    R Vinay Kumar        18.0
## 33  JP Duminy        RG Sharma         3.0
## 34  JP Duminy          S Gopal         8.0
## 35  JP Duminy       SL Malinga         8.0
## 36  JP Duminy       TG Southee         3.0

10. Team batting scorecard(all matches with opposing IPL team)

This function provides the overall scorecard for an IPL team in all matches against another IPL team. In the snippet below the batting scorecard of RCB is show against CSK. Kohli, Gayle and De villiers lead the pack.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Royal Challengers Bangalore-Chennai Super Kings-allMatches.csv")
rcb_csk_matches = pd.read_csv(path)
scorecard=yka.teamBattingScorecardOppnAllMatches(rcb_csk_matches,'Royal Challengers Bangalore',"Chennai Super Kings")
print(scorecard)
##              batsman  runs  balls  4s  6s          SR
## 5            V Kohli   706    570  51  30  123.859649
## 20          CH Gayle   270    228  12  23  118.421053
## 19    AB de Villiers   241    157  26   9  153.503185
## 6           R Dravid   133    117  18   0  113.675214
## 3          JH Kallis   123    113  21   0  108.849558
## 22        MA Agarwal   120    104  15   4  115.384615
## 2        LRPL Taylor   117    102   5   6  114.705882
## 11        RV Uthappa   115     77   7   8  149.350649
## 21         SS Tiwary    86     88   4   3   97.727273
## 17         MK Pandey    73     72  10   0  101.388889
## 32        KD Karthik    61     58   9   0  105.172414
## 34           D Wiese    51     43   4   2  118.604651
## 33           SN Khan    50     36   5   1  138.888889
## 1           W Jaffer    50     36   5   2  138.888889
## 7            P Kumar    39     25   2   2  156.000000
## 28      Yuvraj Singh    38     33   2   1  115.151515
## 4         MV Boucher    37     33   4   1  112.121212
## 23     LA Pomersbach    31     21   2   2  147.619048
## 8             Z Khan    29     27   3   0  107.407407
## 12      KP Pietersen    23     15   2   1  153.333333
## 38          CL White    21     13   2   1  161.538462
## 26       YV Takawale    19     17   4   0  111.764706
## 31          MS Bisla    17     14   3   0  121.428571
## 14     R Vinay Kumar    17     10   1   1  170.000000
## 25        RR Rossouw    15     13   1   1  115.384615
## 40        AUK Pathan    14      6   2   1  233.333333
## 42   JJ van der Wath    14     11   1   1  127.272727
## 27            VH Zol    13     12   0   1  108.333333
## 30          MA Starc    13     16   1   0   81.250000
## 24      MC Henriques    12      4   3   0  300.000000
## 44          A Mithun    11      8   2   0  137.500000
## 50          PA Patel    10     14   2   0   71.428571
## 36        SP Goswami    10     19   1   0   52.631579
## 0           B Chipli     8     12   1   0   66.666667
## 9            B Akhil     8     12   1   0   66.666667
## 29            S Rana     6      8   0   0   75.000000
## 16  RE van der Merwe     5     12   0   0   41.666667
## 49   KB Arun Karthik     5      5   0   0  100.000000
## 54     Mandeep Singh     4      7   0   0   57.142857
## 37     Misbah-ul-Haq     4      6   0   0   66.666667
## 52      NJ Maddinson     4      7   1   0   57.142857
## 51          AN Ahmed     4      1   1   0  400.000000
## 15          A Kumble     3      6   0   0   50.000000
## 43        DL Vettori     3      4   0   0   75.000000
## 47      DT Christian     2      2   0   0  100.000000
## 45   J Syed Mohammad     2      3   0   0   66.666667
## 35          HV Patel     2      5   0   0   40.000000
## 41         CA Pujara     2      6   0   0   33.333333
## 10          DW Steyn     1      5   0   0   20.000000
## 18        EJG Morgan     1      4   0   0   25.000000
## 46        RR Bhatkal     0      2   0   0    0.000000
## 48         R Rampaul     0      6   0   0    0.000000
## 13         R Bishnoi     0      1   0   0    0.000000
## 39        TM Dilshan     0      1   0   0    0.000000
## 53     Iqbal Abdulla     0      3   0   0    0.000000
## 55         S Aravind     0      1   0   0    0.000000

11.Team Bowling scorecard (all matches with opposing IPL team)

The output below gives the performance of Rajasthan Royals bowlers against Kolkata Knight Riders in all matches between the 2 IPL teams.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Kolkata Knight Riders-Rajasthan Royals-allMatches.csv")
rcb_csk_matches = pd.read_csv(path)
scorecard=yka.teamBowlingScorecardOppnAllMatches(rcb_csk_matches,'Rajasthan Royals',"Kolkata Knight Riders")
print(scorecard)
##               bowler  overs  runs  maidens  wicket   econrate
## 31   Shakib Al Hasan     25   153        0       9   6.120000
## 12          I Sharma     15   118        0       6   7.866667
## 33          Umar Gul      8    61        0       6   7.625000
## 29         SP Narine     24   155        0       6   6.458333
## 1           AB Dinda     20   126        0       6   6.300000
## 23     R Vinay Kumar      8    72        0       5   9.000000
## 22          R Bhatia     15   104        0       5   6.933333
## 0         AB Agarkar     12   105        0       4   8.750000
## 17         LR Shukla     12    87        0       4   7.250000
## 6              B Lee     15    90        0       4   6.000000
## 3         AD Russell      7    59        0       4   8.428571
## 34         YK Pathan      8    61        0       4   7.625000
## 14        JD Unadkat      4    26        0       3   6.500000
## 15         JH Kallis     20   149        0       3   7.450000
## 16          L Balaji     11    73        0       3   6.636364
## 27           SE Bond      8    52        1       3   6.500000
## 10     CK Langeveldt      4    15        0       3   3.750000
## 13     Iqbal Abdulla     10    70        0       3   7.000000
## 28   SMSM Senanayake      4    26        0       2   6.500000
## 7         BAW Mendis      4    19        0       2   4.750000
## 18          M Kartik      8    56        0       2   7.000000
## 4      Anureet Singh      4    35        0       2   8.750000
## 32          UT Yadav      7    67        0       2   9.571429
## 30         SS Sarkar      3    15        0       1   5.000000
## 26        SC Ganguly      6    61        0       1  10.166667
## 5      Azhar Mahmood      3    41        0       1  13.666667
## 19          M Morkel      8    78        0       1   9.750000
## 11         DJ Hussey      2    26        0       0  13.000000
## 2         AD Mathews      3    33        0       0  11.000000
## 8           BJ Hodge      2    34        0       0  17.000000
## 25          S Narwal      2    17        0       0   8.500000
## 24  RN ten Doeschate      2    14        0       0   7.000000
## 21         PP Chawla      4    39        0       0   9.750000
## 20    Mohammed Shami      3    26        0       0   8.666667
## 9           CH Gayle      4    20        0       0   5.000000

12. Team Bowling wicket kind -Chart (all matches with opposing IPL team)

The functions compute and display the kind of wickets taken(bowled, caught, lbw etc) by an IPL team in all matches against another IPL team

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Chennai Super Kings-Rajasthan Royals-allMatches.csv")
csk_rr_matches = pd.read_csv(path)
yka.teamBowlingWicketKindOppositionAllMatches(csk_rr_matches,'Chennai Super Kings','Rajasthan Royals',plot=True,top=5,wickets=1)

13. Team Bowling wicket kind -Dataframe (all matches with opposing IPL team)

This gives the type of wickets taken

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Delhi Daredevils-Pune Warriors-allMatches.csv")
dd_pw_matches = pd.read_csv(path)
m=yka.teamBowlingWicketKindOppositionAllMatches(dd_pw_matches,'Pune Warriors','Delhi Daredevils',plot=False,top=4,wickets=1)
print(m)
##       bowler    kind  wickets
## 0  IK Pathan  bowled        1
## 1  IK Pathan  caught        3
## 2   M Morkel  bowled        1
## 3   M Morkel  caught        3
## 4   S Nadeem  bowled        1
## 5   S Nadeem  caught        2
## 6   UT Yadav  caught        3

14 Team Bowler vs Batman -Plot (all matches with opposing IPL team)

The function below gives the performance of bowlers in all matches against another IPL team.

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Sunrisers Hyderabad-Kolkata Knight Riders-allMatches.csv")
srh_kkr_matches = pd.read_csv(path)
yka.teamBowlersVsBatsmenOppnAllMatches(srh_kkr_matches,'Sunrisers Hyderabad','Kolkata Knight Riders',plot=True,top=5,runsConceded=10)

15 Team Bowler vs Batman – Dataframe (all matches with opposing IPL team)

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Royal Challengers Bangalore-Kings XI Punjab-allMatches.csv")
srh_kkr_matches = pd.read_csv(path)
m=yka.teamBowlersVsBatsmenOppnAllMatches(srh_kkr_matches,'Royal Challengers Bangalore','Kings XI Punjab',plot=False,top=1,runsConceded=30)
print(m)
##        bowler           batsman  runsConceded
## 0   PP Chawla          A Kumble             1
## 1   PP Chawla          A Mithun             1
## 2   PP Chawla       AB McDonald             3
## 3   PP Chawla    AB de Villiers            29
## 4   PP Chawla         CA Pujara            13
## 5   PP Chawla          CH Gayle            62
## 6   PP Chawla     CK Langeveldt             1
## 7   PP Chawla          CL White             3
## 8   PP Chawla        DL Vettori             1
## 9   PP Chawla          DT Patil             4
## 10  PP Chawla         JH Kallis            17
## 11  PP Chawla   JJ van der Wath             1
## 12  PP Chawla   KB Arun Karthik             4
## 13  PP Chawla      KP Pietersen            14
## 14  PP Chawla       LRPL Taylor             6
## 15  PP Chawla            M Kaif             2
## 16  PP Chawla         MK Pandey            10
## 17  PP Chawla        MV Boucher             9
## 18  PP Chawla     Misbah-ul-Haq             0
## 19  PP Chawla           P Kumar             0
## 20  PP Chawla          R Dravid            28
## 21  PP Chawla  RE van der Merwe             7
## 22  PP Chawla        RV Uthappa            19
## 23  PP Chawla         SS Tiwary             6
## 24  PP Chawla           V Kohli            56
## 25  PP Chawla            Z Khan             0

16 Team Wins and Losses (all matches with opposing IPL team)

The function below computes and plot the number of wins and losses in a head-on confrontation between 2 IPL teams

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Chennai Super Kings-Delhi Daredevils-allMatches.csv")
csk_dd_matches = pd.read_csv(path)
yka.plotWinLossBetweenTeams(csk_dd_matches,'Chennai Super Kings','Delhi Daredevils')

17 Team Wins by win type (all matches with opposing IPL team)

This function shows how the win happened whether by runs or by wickets

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Chennai Super Kings-Delhi Daredevils-allMatches.csv")
csk_dd_matches = pd.read_csv(path)
yka.plotWinsByRunOrWickets(csk_dd_matches,'Chennai Super Kings')

18 Team Wins by toss decision-field (all matches with opposing IPL team)

This show how Rajasthan Royals fared when it chose to field on winning the toss

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Rajasthan Royals-Kings XI Punjab-allMatches.csv")
rr_kxip_matches = pd.read_csv(path)
yka.plotWinsbyTossDecision(rr_kxip_matches,'Rajasthan Royals',tossDecision='field')

18 Team Wins by toss decision-bat (all matches with opposing IPL team)

This plot shows how Mumbai Indians fared when it chose to bat on winning the toss

import pandas as pd
import os
import yorkpy.analytics as yka
dir1= "C:\\software\\cricket-package\\yorkpyIPLData\\data1"
path=os.path.join(dir1,"Mumbai Indians-Royal Challengers Bangalore-allMatches.csv")
mi_rcb_matches = pd.read_csv(path)
yka.plotWinsbyTossDecision(mi_rcb_matches,'Mumbai Indians',tossDecision='bat')

Feel free to clone/download the code from Github yorkpy

My presentations on ‘Elements of Neural Networks & Deep Learning’ -Parts 6,7,8


This is the final set of presentations in my series ‘Elements of Neural Networks and Deep Learning’. This set follows the earlier 2 sets of presentations namely
1. My presentations on ‘Elements of Neural Networks & Deep Learning’ -Part1,2,3
2. My presentations on ‘Elements of Neural Networks & Deep Learning’ -Parts 4,5

In this final set of presentations I discuss initialization methods, regularization techniques including dropout. Next I also discuss gradient descent optimization methods like momentum, rmsprop, adam etc. Lastly, I briefly also touch on hyper-parameter tuning approaches. The corresponding implementations are available in vectorized R, Python and Octave are available in my book ‘Deep Learning from first principles:Second edition- In vectorized Python, R and Octave

1. Elements of Neural Networks and Deep Learning – Part 6
This part discusses initialization methods specifically like He and Xavier. The presentation also focuses on how to prevent over-fitting using regularization. Lastly the dropout method of regularization is also discusses


The corresponding implementations in vectorized R, Python and Octave of the above discussed methods are available in my post Deep Learning from first principles in Python, R and Octave – Part 6

2. Elements of Neural Networks and Deep Learning – Part 7
This presentation introduces exponentially weighted moving average and shows how this is used in different approaches to gradient descent optimization. The key techniques discussed are learning rate decay, momentum method, rmsprop and adam.


The equivalent implementations of the gradient descent optimization techniques in R, Python and Octave can be seen in my post Deep Learning from first principles in Python, R and Octave – Part 7

3. Elements of Neural Networks and Deep Learning – Part 8
This last part touches upon hyper-parameter tuning in Deep Learning networks


This concludes this series of presentations on “Elements of Neural Networks and Deep Learning’

Checkout my book ‘Deep Learning from first principles: Second Edition – In vectorized Python, R and Octave’. My book starts with the implementation of a simple 2-layer Neural Network and works its way to a generic L-Layer Deep Learning Network, with all the bells and whistles. The derivations have been discussed in detail. The code has been extensively commented and included in its entirety in the Appendix sections. My book is available on Amazon as paperback ($18.99) and and in kindle version($9.99/Rs449).

See also
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. Big Data-1: Move into the big league:Graduate from Python to Pyspark
3. My travels through the realms of Data Science, Machine Learning, Deep Learning and (AI)
4. Revisiting crimes against women in India
5. Introducing cricket package yorkr: Part 1- Beaten by sheer pace!
6. Deblurring with OpenCV: Weiner filter reloaded
7. Taking a closer look at Quantum gates and their operations

To see all posts click Index of posts

My presentations on ‘Elements of Neural Networks & Deep Learning’ -Parts 4,5


This is the next set of presentations on “Elements of Neural Networks and Deep Learning”.  In the 4th presentation I discuss and derive the generalized equations for a multi-unit, multi-layer Deep Learning network.  The 5th presentation derives the equations for a Deep Learning network when performing multi-class classification along with the derivations for cross-entropy loss. The corresponding implementations are available in vectorized R, Python and Octave are available in my book ‘Deep Learning from first principles:Second edition- In vectorized Python, R and Octave

1. Elements of Neural Network and Deep Learning – Part 4
This presentation is a continuation of my 3rd presentation in which I derived the equations for a simple 3 layer Neural Network with 1 hidden layer. In this video presentation, I discuss step-by-step the derivations for a L-Layer, multi-unit Deep Learning Network, with any activation function g(z)


The implementations of L-Layer, multi-unit Deep Learning Network in vectorized R, Python and Octave are available in my post Deep Learning from first principles in Python, R and Octave – Part 3

2. Elements of Neural Network and Deep Learning – Part 5
This presentation discusses multi-class classification using the Softmax function. The detailed derivation for the Jacobian of the Softmax is discussed, and subsequently the derivative of cross-entropy loss is also discussed in detail. Finally the final set of equations for a Neural Network with multi-class classification is derived.


The corresponding implementations in vectorized R, Python and Octave are available in the following posts
a. Deep Learning from first principles in Python, R and Octave – Part 4
b. Deep Learning from first principles in Python, R and Octave – Part 5

To be continued. Watch this space!

Checkout my book ‘Deep Learning from first principles: Second Edition – In vectorized Python, R and Octave’. My book starts with the implementation of a simple 2-layer Neural Network and works its way to a generic L-Layer Deep Learning Network, with all the bells and whistles. The derivations have been discussed in detail. The code has been extensively commented and included in its entirety in the Appendix sections. My book is available on Amazon as paperback ($18.99) and in kindle version($9.99/Rs449).

Also see
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. Big Data-2: Move into the big league:Graduate from R to SparkR
3. Introducing QCSimulator: A 5-qubit quantum computing simulator in R
4. My TEDx talk on the “Internet of Things
5. Rock N’ Roll with Bluemix, Cloudant & NodeExpress
6. GooglyPlus: yorkr analyzes IPL players, teams, matches with plots and tables
7. Literacy in India – A deepR dive
8. Fun simulation of a Chain in Android

To see all posts click Index of Posts

My presentations on ‘Elements of Neural Networks & Deep Learning’ -Part1,2,3


I will be uploading a series of presentations on ‘Elements of Neural Networks and Deep Learning’. In these video presentations I discuss the derivations of L -Layer Deep Learning Networks, starting from the basics. The corresponding implementations are available in vectorized R, Python and Octave are available in my book ‘Deep Learning from first principles:Second edition- In vectorized Python, R and Octave

1. Elements of Neural Networks and Deep Learning – Part 1
This presentation introduces Neural Networks and Deep Learning. A look at history of Neural Networks, Perceptrons and why Deep Learning networks are required and concluding with a simple toy examples of a Neural Network and how they compute

2. Elements of Neural Networks and Deep Learning – Part 2
This presentation takes logistic regression as an example and creates an equivalent 2 layer Neural network. The presentation also takes a look at forward & backward propagation and how the cost is minimized using gradient descent


The implementation of the discussed 2 layer Neural Network in vectorized R, Python and Octave are available in my post ‘Deep Learning from first principles in Python, R and Octave – Part 1

3. Elements of Neural Networks and Deep Learning – Part 3
This 3rd part, discusses a primitive neural network with an input layer, output layer and a hidden layer. The neural network uses tanh activation in the hidden layer and a sigmoid activation in the output layer. The equations for forward and backward propagation are derived.


To see the implementations for the above discussed video see my post ‘Deep Learning from first principles in Python, R and Octave – Part 2

To be continued. Watch this space!

Checkout my book ‘Deep Learning from first principles: Second Edition – In vectorized Python, R and Octave’. My book starts with the implementation of a simple 2-layer Neural Network and works its way to a generic L-Layer Deep Learning Network, with all the bells and whistles. The derivations have been discussed in detail. The code has been extensively commented and included in its entirety in the Appendix sections. My book is available on Amazon as paperback ($18.99) and in kindle version($9.99/Rs449).

You may also like
1. My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
2. Introducing cricpy:A python package to analyze performances of cricketers
3. Natural language processing: What would Shakespeare say?
4. TWS-4: Gossip protocol: Epidemics and rumors to the rescue
5. Getting started with memcached-libmemcached
6. Simplifying ML: Impact of degree of polynomial degree on bias & variance and other insights

To see all posts click Index of posts

My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon


Are you wondering whether to get into the ‘R’ bus or ‘Python’ bus?
My suggestion is to you is “Why not get into the ‘R and Python’ train?”

The third edition of my book ‘Practical Machine Learning with R and Python – Machine Learning in stereo’ is now available in both paperback ($12.99) and kindle ($8.99/Rs449) versions.  In the third edition all code sections have been re-formatted to use the fixed width font ‘Consolas’. This neatly organizes output which have columns like confusion matrix, dataframes etc to be columnar, making the code more readable.  There is a science to formatting too!! which improves the look and feel. It is little wonder that Steve Jobs had a keen passion for calligraphy! Additionally some typos have been fixed.

 

In this book I implement some of the most common, but important Machine Learning algorithms in R and equivalent Python code.
1. Practical machine with R and Python: Third Edition – Machine Learning in Stereo(Paperback-$12.99)
2. Practical machine with R and Python Third Edition – Machine Learning in Stereo(Kindle- $8.99/Rs449)

This book is ideal both for beginners and the experts in R and/or Python. Those starting their journey into datascience and ML will find the first 3 chapters useful, as they touch upon the most important programming constructs in R and Python and also deal with equivalent statements in R and Python. Those who are expert in either of the languages, R or Python, will find the equivalent code ideal for brushing up on the other language. And finally,those who are proficient in both languages, can use the R and Python implementations to internalize the ML algorithms better.

Here is a look at the topics covered

Table of Contents
Preface …………………………………………………………………………….4
Introduction ………………………………………………………………………6
1. Essential R ………………………………………………………………… 8
2. Essential Python for Datascience ……………………………………………57
3. R vs Python …………………………………………………………………81
4. Regression of a continuous variable ……………………………………….101
5. Classification and Cross Validation ………………………………………..121
6. Regression techniques and regularization ………………………………….146
7. SVMs, Decision Trees and Validation curves ………………………………191
8. Splines, GAMs, Random Forests and Boosting ……………………………222
9. PCA, K-Means and Hierarchical Clustering ………………………………258
References ……………………………………………………………………..269

Pick up your copy today!!
Hope you have a great time learning as I did while implementing these algorithms!

Pitching yorkpy … short of good length to IPL – Part 1


I fear not the man who has practiced 10,000 kicks once, but I fear the man who has practiced one kick 10,000 times.
Bruce Lee

I’ve missed more than 9000 shots in my career. I’ve lost almost 300 games. 26 times, I’ve been trusted to take the game winning shot and missed. I’ve failed over and over and over again in my life. And that is why I succeed.
Michael Jordan

Man, it doesn’t matter where you come in to bat, the score is still zero
Viv Richards

Introduction

“If cricketr is to cricpy, then yorkr is to _____?”. Yes, you guessed it right, it is yorkpy. In this post, I introduce my 2nd python package, yorkpy, which is a python clone of my R package yorkr. This package is based on data from Cricsheet. yorkpy currently handles IPL T20 matches.

When I created cricpy, the python avatar, of my R package cricketr, see Introducing cricpy:A python package to analyze performances of cricketers, I had decided that I should avoid doing a python avatar of my R package yorkr (see Introducing cricket package yorkr: Part 1- Beaten by sheer pace!) , as it was more involved, and required the parsing of match data available as yaml files.

Just out of curiosity, I tried the python package ‘yaml’ to read the match data, and lo and behold, I was sucked into the developing the package and so, yorkpy was born. Of course, it goes without saying that, usually when I am in the thick of developing something, I occasionally wonder, why I am doing it, for whom and for what purpose? Maybe it is the joy of ideation, the problem-solving,  the programmer’s high, for sharing my ideas etc. Anyway, whatever be the reason, I hope you enjoy this post and also find yorkpy useful.

You can clone/download the code at Github yorkpy
This post has been published to RPubs at yorkpy-Part1
You can download this post as PDF at IPLT20-yorkpy-part1

The IPL T20 functions in yorkpy are

2. Install the package using ‘pip install’

import pandas as pd
import yorkpy.analytics as yka
#pip install yorkpy

3. Load a yaml file from Cricsheet

There are 2 functions that can be to convert the IPL Twenty20 yaml files to pandas dataframeare

  1. convertYaml2PandasDataframeT20
  2. convertAllYaml2PandasDataframesT20

Note 1: While I have already converted the IPL T20 files, you will need to use these functions for future IPL matches

4. Convert and save IPL T20 yaml file to pandas dataframe

This function will convert a IPL T20 IPL yaml file, in the format as specified in Cricsheet to pandas dataframe. This will be saved as as CSV file in the target directory. The name of the file wil have the following format team1-team2-date.csv. The IPL T20 zip file can be downloaded from Indian Premier League matches.  An example of how a yaml file can be converted to a dataframe and saved is shown below.

import pandas as pd
import yorkpy.analytics as yka
#convertYaml2PandasDataframe(".\\1082593.yaml","..\ipl", ..\\data")

5. Convert and save all IPL T20 yaml files to dataframes

This function will convert all IPL T20 yaml files from a source directory to dataframes, and save it in the target directory, with the names as mentioned above. Since I have already done this, I will not be executing this again. You can download the zip of all the converted RData files from Github at yorkpyData

import pandas as pd
import yorkpy.analytics as yka
#convertAllYaml2PandasDataframes("..\\ipl", "..\\data")

You can download the the zip of the files and use it directly in the functions as follows.For the analysis below I chosen a set of random IPL matches

The randomly selected IPL T20 matches are

  • Chennai Super Kings vs Kings Xi Punjab, 2014-05-30
  • Deccan Chargers vs Delhi Daredevils, 2012-05-10
  • Gujarat Lions vs Mumbai Indians, 2017-04-29
  • Kolkata Knight Riders vs Rajasthan Royals, 2010-04-17
  • Rising Pune Supergiants vs Royal Challengers Bangalore, 2017-04-29

6. Team batting scorecard

The function below computes the batting score card of a team in an IPL match. The scorecard gives the balls faced, the runs scored, 4s, 6s and strike rate. The example below is based on the CSK KXIP match on 30 May 2014.

You can check against the actual scores in this match Chennai Super Kings-Kings XI Punjab-2014-05-30

import pandas as pd
import yorkpy.analytics as yka
csk_kxip=pd.read_csv(".\\Chennai Super Kings-Kings XI Punjab-2014-05-30.csv")
scorecard,extras=yka.teamBattingScorecardMatch(csk_kxip,"Chennai Super Kings")
print(scorecard)
##         batsman  runs  balls  4s  6s          SR
## 0      DR Smith     7     12   0   0   58.333333
## 1  F du Plessis     0      1   0   0    0.000000
## 2      SK Raina    87     26  12   6  334.615385
## 3   BB McCullum    11     16   0   0   68.750000
## 4     RA Jadeja    27     22   2   1  122.727273
## 5     DJ Hussey     1      3   0   0   33.333333
## 6      MS Dhoni    42     34   3   3  123.529412
## 7      R Ashwin    10     11   0   0   90.909091
## 8     MM Sharma     1      3   0   0   33.333333
print(extras)
##    total  wides  noballs  legbyes  byes  penalty  extras
## 0    428     14        3        5     5        0      27
print("\n\n")
scorecard1,extras1=yka.teamBattingScorecardMatch(csk_kxip,"Kings XI Punjab")
print(scorecard1)
##       batsman  runs  balls  4s  6s          SR
## 0    V Sehwag   122     62  12   8  196.774194
## 1     M Vohra    34     33   1   2  103.030303
## 2  GJ Maxwell    13      8   1   1  162.500000
## 3   DA Miller    38     19   5   1  200.000000
## 4   GJ Bailey     1      2   0   0   50.000000
## 5     WP Saha     6      4   0   1  150.000000
## 6  MG Johnson     1      1   0   0  100.000000
print(extras1)
##    total  wides  noballs  legbyes  byes  penalty  extras
## 0    428     14        3        5     5        0      27

Let’s take another random match between Gujarat Lions and Mumbai Indian on 29 Apr 2017 Gujarat Lions-Mumbai Indians-2017-04-29

import pandas as pd
gl_mi=pd.read_csv(".\\Gujarat Lions-Mumbai Indians-2017-04-29.csv")
import yorkpy.analytics as yka
scorecard,extras=yka.teamBattingScorecardMatch(gl_mi,"Gujarat Lions")
print(scorecard)
##          batsman  runs  balls  4s  6s          SR
## 0   Ishan Kishan    48     38   6   2  126.315789
## 1    BB McCullum     6      4   1   0  150.000000
## 2       SK Raina     1      3   0   0   33.333333
## 3       AJ Finch     0      3   0   0    0.000000
## 4     KD Karthik     2      9   0   0   22.222222
## 5      RA Jadeja    28     22   2   1  127.272727
## 6    JP Faulkner    21     29   2   0   72.413793
## 7      IK Pathan     2      3   0   0   66.666667
## 8         AJ Tye    25     12   2   2  208.333333
## 9   Basil Thampi     2      4   0   0   50.000000
## 10    Ankit Soni     7      2   0   1  350.000000
print(extras)
##    total  wides  noballs  legbyes  byes  penalty  extras
## 0    306      8        3        1     0        0      12
print("\n\n")
scorecard1,extras1=yka.teamBattingScorecardMatch(gl_mi,"Mumbai Indians")
print(scorecard1)
##             batsman  runs  balls  4s  6s          SR
## 0          PA Patel    70     45   9   1  155.555556
## 1        JC Buttler     9      7   2   0  128.571429
## 2            N Rana    19     16   1   1  118.750000
## 3         RG Sharma     5     13   0   0   38.461538
## 4        KA Pollard    15     11   2   0  136.363636
## 5         KH Pandya    29     20   2   1  145.000000
## 6         HH Pandya     4      5   0   0   80.000000
## 7   Harbhajan Singh     0      1   0   0    0.000000
## 8    MJ McClenaghan     1      1   0   0  100.000000
## 9         JJ Bumrah     0      1   0   0    0.000000
## 10       SL Malinga     0      1   0   0    0.000000
print(extras1)
##    total  wides  noballs  legbyes  byes  penalty  extras
## 0    306      8        3        1     0        0      12

7. Plot the team batting partnerships

The functions below plot the team batting partnership in the match. It shows what the partnership were in the mtach

Note: Many of the plots include an additional parameters plot which is either True or False. The default value is plot=True. When plot=True the plot will be displayed. When plot=False the data frame will be returned to the user. The user can use this to create an interactive chart using one of the packages like rcharts, ggvis,googleVis or plotly.

import pandas as pd
import yorkpy.analytics as yka
dc_dd=pd.read_csv(".\\Deccan Chargers-Delhi Daredevils-2012-05-10.csv")
yka.teamBatsmenPartnershipMatch(dc_dd,'Deccan Chargers','Delhi Daredevils')

yka.teamBatsmenPartnershipMatch(dc_dd,'Delhi Daredevils','Deccan Chargers',plot=True)
# Print partnerships as a dataframe

rps_rcb=pd.read_csv(".\\Rising Pune Supergiant-Royal Challengers Bangalore-2017-04-29.csv")
m=yka.teamBatsmenPartnershipMatch(rps_rcb,'Royal Challengers Bangalore','Rising Pune Supergiant',plot=False)
print(m)
##            batsman     non_striker  runs
## 0   AB de Villiers         V Kohli     3
## 1         AF Milne         V Kohli     5
## 2        KM Jadhav         V Kohli     7
## 3           P Negi         V Kohli     3
## 4        S Aravind         V Kohli     0
## 5        S Aravind       YS Chahal     8
## 6         S Badree         V Kohli     2
## 7        STR Binny         V Kohli     1
## 8      Sachin Baby         V Kohli     2
## 9          TM Head         V Kohli     2
## 10         V Kohli  AB de Villiers    17
## 11         V Kohli        AF Milne     5
## 12         V Kohli       KM Jadhav     4
## 13         V Kohli          P Negi     9
## 14         V Kohli       S Aravind     2
## 15         V Kohli        S Badree     8
## 16         V Kohli     Sachin Baby     1
## 17         V Kohli         TM Head     9
## 18       YS Chahal       S Aravind     4

8. Batsmen vs Bowler

The function below computes and plots the performances of the batsmen vs the bowlers. As before the plot parameter can be set to True or False. By default it is plot=True

import pandas as pd
import yorkpy.analytics as yka
gl_mi=pd.read_csv(".\\Gujarat Lions-Mumbai Indians-2017-04-29.csv")
yka.teamBatsmenVsBowlersMatch(gl_mi,"Gujarat Lions","Mumbai Indians", plot=True)
# Print 

csk_kxip=pd.read_csv(".\\Chennai Super Kings-Kings XI Punjab-2014-05-30.csv")
m=yka.teamBatsmenVsBowlersMatch(csk_kxip,'Chennai Super Kings','Kings XI Punjab',plot=False)
print(m)
##          batsman           bowler  runs
## 0    BB McCullum         AR Patel     4
## 1    BB McCullum       GJ Maxwell     1
## 2    BB McCullum  Karanveer Singh     6
## 3      DJ Hussey          P Awana     1
## 4       DR Smith       MG Johnson     7
## 5       DR Smith          P Awana     0
## 6       DR Smith   Sandeep Sharma     0
## 7   F du Plessis       MG Johnson     0
## 8      MM Sharma         AR Patel     0
## 9      MM Sharma       MG Johnson     0
## 10     MM Sharma          P Awana     1
## 11      MS Dhoni         AR Patel    12
## 12      MS Dhoni  Karanveer Singh     2
## 13      MS Dhoni       MG Johnson    11
## 14      MS Dhoni          P Awana    15
## 15      MS Dhoni   Sandeep Sharma     2
## 16      R Ashwin         AR Patel     1
## 17      R Ashwin  Karanveer Singh     4
## 18      R Ashwin       MG Johnson     1
## 19      R Ashwin          P Awana     1
## 20      R Ashwin   Sandeep Sharma     3
## 21     RA Jadeja         AR Patel     5
## 22     RA Jadeja       GJ Maxwell     3
## 23     RA Jadeja  Karanveer Singh    19
## 24     RA Jadeja          P Awana     0
## 25      SK Raina       MG Johnson    21
## 26      SK Raina          P Awana    40
## 27      SK Raina   Sandeep Sharma    26

9. Bowling Scorecard

This function provides the bowling performance, the number of overs bowled, maidens, runs conceded. wickets taken and economy rate for the IPL match

import pandas as pd
import yorkpy.analytics as yka
dc_dd=pd.read_csv(".\\Deccan Chargers-Delhi Daredevils-2012-05-10.csv")
a=yka.teamBowlingScorecardMatch(dc_dd,'Deccan Chargers')
print(a)
##        bowler  overs  runs  maidens  wicket  econrate
## 0  AD Russell      4    39        0       0      9.75
## 1   IK Pathan      4    46        0       1     11.50
## 2    M Morkel      4    32        0       1      8.00
## 3    S Nadeem      4    39        0       0      9.75
## 4    VR Aaron      4    30        0       2      7.50
rps_rcb=pd.read_csv(".\\Rising Pune Supergiant-Royal Challengers Bangalore-2017-04-29.csv")
b=yka.teamBowlingScorecardMatch(rps_rcb,'Royal Challengers Bangalore')
print(b)
##               bowler  overs  runs  maidens  wicket  econrate
## 0          DL Chahar      2    18        0       0      9.00
## 1       DT Christian      4    25        0       1      6.25
## 2        Imran Tahir      4    18        0       3      4.50
## 3         JD Unadkat      4    19        0       1      4.75
## 4        LH Ferguson      4     7        1       3      1.75
## 5  Washington Sundar      2     7        0       1      3.50

10. Wicket Kind

The plots below provide the kind of wicket taken by the bowler (caught, bowled, lbw etc.) for the IPL match

import pandas as pd
import yorkpy.analytics as yka
kkr_rr=pd.read_csv(".\\Kolkata Knight Riders-Rajasthan Royals-2010-04-17.csv")
yka.teamBowlingWicketKindMatch(kkr_rr,'Kolkata Knight Riders','Rajasthan Royals')

csk_kxip=pd.read_csv(".\\Chennai Super Kings-Kings XI Punjab-2014-05-30.csv")
m = yka.teamBowlingWicketKindMatch(csk_kxip,'Chennai Super Kings','Kings-Kings XI Punjab',plot=False)
print(m)
##             bowler     kind  player_out
## 0         AR Patel  run out           1
## 1         AR Patel  stumped           1
## 2  Karanveer Singh  run out           1
## 3       MG Johnson   caught           1
## 4          P Awana   caught           2
## 5   Sandeep Sharma   bowled           1

11. Wicket vs Runs conceded

The plots below provide the wickets taken and the runs conceded by the bowler in the IPL T20 match

import pandas as pd
import yorkpy.analytics as yka
dc_dd=pd.read_csv(".\\Deccan Chargers-Delhi Daredevils-2012-05-10.csv")
yka.teamBowlingWicketMatch(dc_dd,"Deccan Chargers", "Delhi Daredevils",plot=True)

print("\n\n")
rps_rcb=pd.read_csv(".\\Rising Pune Supergiant-Royal Challengers Bangalore-2017-04-29.csv")
a=yka.teamBowlingWicketMatch(rps_rcb,"Royal Challengers Bangalore", "Rising Pune Supergiant",plot=False)
print(a)
##               bowler      player_out  kind
## 0       DT Christian         V Kohli     1
## 1        Imran Tahir        AF Milne     1
## 2        Imran Tahir          P Negi     1
## 3        Imran Tahir        S Badree     1
## 4         JD Unadkat         TM Head     1
## 5        LH Ferguson  AB de Villiers     1
## 6        LH Ferguson       KM Jadhav     1
## 7        LH Ferguson       STR Binny     1
## 8  Washington Sundar     Sachin Baby     1

12. Bowler Vs Batsmen

The functions compute and display how the different bowlers of the IPL team performed against the batting opposition.

import pandas as pd
import yorkpy.analytics as yka
csk_kxip=pd.read_csv(".\\Chennai Super Kings-Kings XI Punjab-2014-05-30.csv")
yka.teamBowlersVsBatsmenMatch(csk_kxip,"Chennai Super Kings","Kings XI Punjab")

print("\n\n")
kkr_rr=pd.read_csv(".\\Kolkata Knight Riders-Rajasthan Royals-2010-04-17.csv")
m =yka.teamBowlersVsBatsmenMatch(kkr_rr,"Rajasthan Royals","Kolkata Knight Riders",plot=False)
print(m)
##        batsman      bowler  runs
## 0     AC Voges    AB Dinda     1
## 1     AC Voges  JD Unadkat     1
## 2     AC Voges   LR Shukla     1
## 3     AC Voges    M Kartik     5
## 4     AJ Finch    AB Dinda     3
## 5     AJ Finch  JD Unadkat     3
## 6     AJ Finch   LR Shukla    13
## 7     AJ Finch    M Kartik     2
## 8     AJ Finch     SE Bond     0
## 9      AS Raut    AB Dinda     1
## 10     AS Raut  JD Unadkat     1
## 11    FY Fazal    AB Dinda     1
## 12    FY Fazal   LR Shukla     3
## 13    FY Fazal    M Kartik     3
## 14    FY Fazal     SE Bond     6
## 15     NV Ojha    AB Dinda    10
## 16     NV Ojha  JD Unadkat     5
## 17     NV Ojha   LR Shukla     0
## 18     NV Ojha    M Kartik     1
## 19     NV Ojha     SE Bond     2
## 20     P Dogra  JD Unadkat     2
## 21     P Dogra   LR Shukla     5
## 22     P Dogra    M Kartik     1
## 23     P Dogra     SE Bond     0
## 24  SK Trivedi    AB Dinda     4
## 25    SK Warne    AB Dinda     2
## 26    SK Warne    M Kartik     1
## 27    SK Warne     SE Bond     0
## 28   SR Watson    AB Dinda     2
## 29   SR Watson  JD Unadkat    13
## 30   SR Watson   LR Shukla     1
## 31   SR Watson    M Kartik    18
## 32   SR Watson     SE Bond    10
## 33   YK Pathan  JD Unadkat     1
## 34   YK Pathan   LR Shukla     7

13. Match worm chart

The plots below provide the match worm graph for the IPL Twenty 20 matches

import pandas as pd
import yorkpy.analytics as yka
dc_dd=pd.read_csv(".\\Deccan Chargers-Delhi Daredevils-2012-05-10.csv")
yka.matchWormChart(dc_dd,"Deccan Chargers", "Delhi Daredevils")

gl_mi=pd.read_csv(".\\Gujarat Lions-Mumbai Indians-2017-04-29.csv")
yka.matchWormChart(gl_mi,"Mumbai Indians","Gujarat Lions")

Feel free to clone/download the code from Github yorkpy

Conclusion

This post included all functions between 2 IPL teams from the package yorkpy for IPL Twenty20 matches. As mentioned above the yaml match files have been already converted to dataframes and are available for download from Github at yorkpyData

After having used Python and R for analytics, Machine Learning and Deep Learning, I have now realized that neither language is superior or inferior. Both have, some good packages and some that are not so well suited.

To be continued. Watch this space!

You may also like
1.My book ‘Deep Learning from first principles:Second Edition’ now on Amazon
2.My book ‘Practical Machine Learning in R and Python: Second edition’ on Amazon
2. Cricpy takes a swing at the ODIs
3. Introducing cricket package yorkr: Part 1- Beaten by sheer pace!
4. Big Data-1: Move into the big league:Graduate from Python to Pyspark
5. Simulating an Edge Shape in Android

To see all posts click Index of posts

Analyzing batsmen and bowlers with cricpy template


Introduction

This post shows how you can analyze batsmen and bowlers of Test, ODI and T20s using cricpy templates, using data from ESPN Cricinfo.

The cricpy package

The data for a particular player can be obtained with the getPlayerData() function. To do you will need to go to ESPN CricInfo Player and type in the name of the player for e.g Rahul Dravid, Virat Kohli  etc. This will bring up a page which have the profile number for the player e.g. for Rahul Dravid this would be http://www.espncricinfo.com/india/content/player/28114.html. Hence, Dravid’s profile is 28114. This can be used to get the data for Rahul Dravid as shown below

1. For Test players use batting and bowling.
2. For ODI use batting and bowling
3. For T20 use T20 Batting T20 Bowling

Please mindful of the  ESPN Cricinfo Terms of Use

My posts on Cripy were
a. Introducing cricpy:A python package to analyze performances of cricketers
b. Cricpy takes a swing at the ODIs
c. Cricpy takes guard for the Twenty20s

You can clone/download this cricpy template for your own analysis of players. This can be done using RStudio or IPython notebooks from Github at cricpy-template. You can uncomment the functions and use them.

The cricpy package is now available with pip install cricpy!!!

1 Importing cricpy – Python

# Install the package
# Do a pip install cricpy
# Import cricpy
import cricpy.analytics as ca 
## C:\Users\Ganesh\ANACON~1\lib\site-packages\statsmodels\compat\pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
##   from pandas.core import datetools

2. Invoking functions with Python package cricpy

import cricpy.analytics as ca 
#ca.batsman4s("aplayer.csv","A Player")

3. Getting help from cricpy – Python

import cricpy.analytics as ca
#help(ca.getPlayerData)

The details below will introduce the different functions that are available in cricpy.

4. Get the player data for a player using the function getPlayerData()

Important Note This needs to be done only once for a player. This function stores the player’s data in the specified CSV file (for e.g. dravid.csv as above) which can then be reused for all other functions). Once we have the data for the players many analyses can be done. This post will use the stored CSV file obtained with a prior getPlayerData for all subsequent analyses

4a. For Test players

import cricpy.analytics as ca
#player1 =ca.getPlayerData(profileNo1,dir="..",file="player1.csv",type="batting",homeOrAway=[1,2], result=[1,2,4])
#player1 =ca.getPlayerData(profileNo2,dir="..",file="player2.csv",type="batting",homeOrAway=[1,2], result=[1,2,4])

4b. For ODI players

import cricpy.analytics as ca
#player1 =ca.getPlayerDataOD(profileNo1,dir="..",file="player1.csv",type="batting")
#player1 =ca.getPlayerDataOD(profileNo2,dir="..",file="player2.csv",type="batting"")

4c For T20 players

import cricpy.analytics as ca
#player1 =ca.getPlayerDataTT(profileNo1,dir="..",file="player1.csv",type="batting")
#player1 =ca.getPlayerDataTT(profileNo2,dir="..",file="player2.csv",type="batting"")

5 A Player’s performance – Basic Analyses

The 3 plots below provide the following for Rahul Dravid

  1. Frequency percentage of runs in each run range over the whole career
  2. Mean Strike Rate for runs scored in the given range
  3. A histogram of runs frequency percentages in runs ranges
import cricpy.analytics as ca
import matplotlib.pyplot as plt
#ca.batsmanRunsFreqPerf("aplayer.csv","A Player")
#ca.batsmanMeanStrikeRate("aplayer.csv","A Player")
#ca.batsmanRunsRanges("aplayer.csv","A Player") 

6. More analyses

This gives details on the batsmen’s 4s, 6s and dismissals

import cricpy.analytics as ca
#ca.batsman4s("aplayer.csv","A Player")
#ca.batsman6s("aplayer.csv","A Player") 
#ca.batsmanDismissals("aplayer.csv","A Player")
# The below function is for ODI and T20 only
#ca.batsmanScoringRateODTT("./kohli.csv","Virat Kohli")  

7. 3D scatter plot and prediction plane

The plots below show the 3D scatter plot of Runs versus Balls Faced and Minutes at crease. A linear regression plane is then fitted between Runs and Balls Faced + Minutes at crease

import cricpy.analytics as ca
#ca.battingPerf3d("aplayer.csv","A Player")

8. Average runs at different venues

The plot below gives the average runs scored at different grounds. The plot also the number of innings at each ground as a label at x-axis.

import cricpy.analytics as ca
#ca.batsmanAvgRunsGround("aplayer.csv","A Player")

9. Average runs against different opposing teams

This plot computes the average runs scored against different countries.

import cricpy.analytics as ca
#ca.batsmanAvgRunsOpposition("aplayer.csv","A Player")

10. Highest Runs Likelihood

The plot below shows the Runs Likelihood for a batsman.

import cricpy.analytics as ca
#ca.batsmanRunsLikelihood("aplayer.csv","A Player")

11. A look at the Top 4 batsman

Choose any number of players

1.Player1 2.Player2 3.Player3 …

The following plots take a closer at their performances. The box plots show the median the 1st and 3rd quartile of the runs

12. Box Histogram Plot

This plot shows a combined boxplot of the Runs ranges and a histogram of the Runs Frequency

import cricpy.analytics as ca
#ca.batsmanPerfBoxHist("aplayer001.csv","A Player001")
#ca.batsmanPerfBoxHist("aplayer002.csv","A Player002")
#ca.batsmanPerfBoxHist("aplayer003.csv","A Player003")
#ca.batsmanPerfBoxHist("aplayer004.csv","A Player004")

13. Get Player Data special

import cricpy.analytics as ca
#player1sp = ca.getPlayerDataSp(profile1,tdir=".",tfile="player1sp.csv",ttype="batting")
#player2sp = ca.getPlayerDataSp(profile2,tdir=".",tfile="player2sp.csv",ttype="batting")
#player3sp = ca.getPlayerDataSp(profile3,tdir=".",tfile="player3sp.csv",ttype="batting")
#player4sp = ca.getPlayerDataSp(profile4,tdir=".",tfile="player4sp.csv",ttype="batting")

14. Contribution to won and lost matches

Note:This can only be used for Test matches

import cricpy.analytics as ca
#ca.batsmanContributionWonLost("player1sp.csv","A Player001")
#ca.batsmanContributionWonLost("player2sp.csv","A Player002")
#ca.batsmanContributionWonLost("player3sp.csv","A Player003")
#ca.batsmanContributionWonLost("player4sp.csv","A Player004")

15. Performance at home and overseas

Note:This can only be used for Test matches This function also requires the use of getPlayerDataSp() as shown above

import cricpy.analytics as ca
#ca.batsmanPerfHomeAway("player1sp.csv","A Player001")
#ca.batsmanPerfHomeAway("player2sp.csv","A Player002")
#ca.batsmanPerfHomeAway("player3sp.csv","A Player003")
#ca.batsmanPerfHomeAway("player4sp.csv","A Player004")

16 Moving Average of runs in career

import cricpy.analytics as ca
#ca.batsmanMovingAverage("aplayer001.csv","A Player001")
#ca.batsmanMovingAverage("aplayer002.csv","A Player002")
#ca.batsmanMovingAverage("aplayer003.csv","A Player003")
#ca.batsmanMovingAverage("aplayer004.csv","A Player004")

17 Cumulative Average runs of batsman in career

This function provides the cumulative average runs of the batsman over the career.

import cricpy.analytics as ca
#ca.batsmanCumulativeAverageRuns("aplayer001.csv","A Player001")
#ca.batsmanCumulativeAverageRuns("aplayer002.csv","A Player002")
#ca.batsmanCumulativeAverageRuns("aplayer003.csv","A Player003")
#ca.batsmanCumulativeAverageRuns("aplayer004.csv","A Player004")

18 Cumulative Average strike rate of batsman in career

.

import cricpy.analytics as ca
#ca.batsmanCumulativeStrikeRate("aplayer001.csv","A Player001")
#ca.batsmanCumulativeStrikeRate("aplayer002.csv","A Player002")
#ca.batsmanCumulativeStrikeRate("aplayer003.csv","A Player003")
#ca.batsmanCumulativeStrikeRate("aplayer004.csv","A Player004")

19 Future Runs forecast

import cricpy.analytics as ca
#ca.batsmanPerfForecast("aplayer001.csv","A Player001")

20 Relative Batsman Cumulative Average Runs

The plot below compares the Relative cumulative average runs of the batsman for each of the runs ranges of 10 and plots them.

import cricpy.analytics as ca
frames = ["aplayer1.csv","aplayer2.csv","aplayer3.csv","aplayer4.csv"]
names = ["A Player1","A Player2","A Player3","A Player4"]
#ca.relativeBatsmanCumulativeAvgRuns(frames,names)

21 Plot of 4s and 6s

import cricpy.analytics as ca
frames = ["aplayer1.csv","aplayer2.csv","aplayer3.csv","aplayer4.csv"]
names = ["A Player1","A Player2","A Player3","A Player4"]
#ca.batsman4s6s(frames,names)

22. Relative Batsman Strike Rate

The plot below gives the relative Runs Frequency Percetages for each 10 run bucket. The plot below show

import cricpy.analytics as ca
frames = ["aplayer1.csv","aplayer2.csv","aplayer3.csv","aplayer4.csv"]
names = ["A Player1","A Player2","A Player3","A Player4"]
#ca.relativeBatsmanCumulativeStrikeRate(frames,names)

23. 3D plot of Runs vs Balls Faced and Minutes at Crease

The plot is a scatter plot of Runs vs Balls faced and Minutes at Crease. A prediction plane is fitted

import cricpy.analytics as ca
#ca.battingPerf3d("aplayer001.csv","A Player001")
#ca.battingPerf3d("aplayer002.csv","A Player002")
#ca.battingPerf3d("aplayer003.csv","A Player003")
#ca.battingPerf3d("aplayer004.csv","A Player004")

24. Predicting Runs given Balls Faced and Minutes at Crease

A multi-variate regression plane is fitted between Runs and Balls faced +Minutes at crease.

import cricpy.analytics as ca
import numpy as np
import pandas as pd
BF = np.linspace( 10, 400,15)
Mins = np.linspace( 30,600,15)
newDF= pd.DataFrame({'BF':BF,'Mins':Mins})
#aplayer = ca.batsmanRunsPredict("aplayer.csv",newDF,"A Player")
#print(aplayer)

The fitted model is then used to predict the runs that the batsmen will score for a given Balls faced and Minutes at crease.

25 Analysis of Top 3 wicket takers

Take any number of bowlers from either Test, ODI or T20

  1. Bowler1
  2. Bowler2
  3. Bowler3 …

26. Get the bowler’s data (Test)

This plot below computes the percentage frequency of number of wickets taken for e.g 1 wicket x%, 2 wickets y% etc and plots them as a continuous line

import cricpy.analytics as ca
#abowler1 =ca.getPlayerData(profileNo1,dir=".",file="abowler1.csv",type="bowling",homeOrAway=[1,2], result=[1,2,4])
#abowler2 =ca.getPlayerData(profileNo2,dir=".",file="abowler2.csv",type="bowling",homeOrAway=[1,2], result=[1,2,4])
#abowler3 =ca.getPlayerData(profile3,dir=".",file="abowler3.csv",type="bowling",homeOrAway=[1,2], result=[1,2,4])

26b For ODI bowlers

import cricpy.analytics as ca
#abowler1 =ca.getPlayerDataOD(profileNo1,dir=".",file="abowler1.csv",type="bowling")
#abowler2 =ca.getPlayerDataOD(profileNo2,dir=".",file="abowler2.csv",type="bowling")
#abowler3 =ca.getPlayerDataOD(profile3,dir=".",file="abowler3.csv",type="bowling")

26c For T20 bowlers

import cricpy.analytics as ca
#abowler1 =ca.getPlayerDataTT(profileNo1,dir=".",file="abowler1.csv",type="bowling")
#abowler2 =ca.getPlayerDataTT(profileNo2,dir=".",file="abowler2.csv",type="bowling")
#abowler3 =ca.getPlayerDataTT(profile3,dir=".",file="abowler3.csv",type="bowling")

27. Wicket Frequency Plot

This plot below plots the frequency of wickets taken for each of the bowlers

import cricpy.analytics as ca
#ca.bowlerWktsFreqPercent("abowler1.csv","A Bowler1")
#ca.bowlerWktsFreqPercent("abowler2.csv","A Bowler2")
#ca.bowlerWktsFreqPercent("abowler3.csv","A Bowler3")

28. Wickets Runs plot

The plot below create a box plot showing the 1st and 3rd quartile of runs conceded versus the number of wickets taken

import cricpy.analytics as ca
#ca.bowlerWktsRunsPlot("abowler1.csv","A Bowler1")
#ca.bowlerWktsRunsPlot("abowler2.csv","A Bowler2")
#ca.bowlerWktsRunsPlot("abowler3.csv","A Bowler3")

29 Average wickets at different venues

The plot gives the average wickets taken bat different venues.

import cricpy.analytics as ca
#ca.bowlerAvgWktsGround("abowler1.csv","A Bowler1")
#ca.bowlerAvgWktsGround("abowler2.csv","A Bowler2")
#ca.bowlerAvgWktsGround("abowler3.csv","A Bowler3")

30 Average wickets against different opposition

The plot gives the average wickets taken against different countries.

import cricpy.analytics as ca
#ca.bowlerAvgWktsOpposition("abowler1.csv","A Bowler1")
#ca.bowlerAvgWktsOpposition("abowler2.csv","A Bowler2")
#ca.bowlerAvgWktsOpposition("abowler3.csv","A Bowler3")

31 Wickets taken moving average

import cricpy.analytics as ca
#ca.bowlerMovingAverage("abowler1.csv","A Bowler1")
#ca.bowlerMovingAverage("abowler2.csv","A Bowler2")
#ca.bowlerMovingAverage("abowler3.csv","A Bowler3")

32 Cumulative average wickets taken

The plots below give the cumulative average wickets taken by the bowlers.

import cricpy.analytics as ca
#ca.bowlerCumulativeAvgWickets("abowler1.csv","A Bowler1")
#ca.bowlerCumulativeAvgWickets("abowler2.csv","A Bowler2")
#ca.bowlerCumulativeAvgWickets("abowler3.csv","A Bowler3")

33 Cumulative average economy rate

The plots below give the cumulative average economy rate of the bowlers.

import cricpy.analytics as ca
#ca.bowlerCumulativeAvgEconRate("abowler1.csv","A Bowler1")
#ca.bowlerCumulativeAvgEconRate("abowler2.csv","A Bowler2")
#ca.bowlerCumulativeAvgEconRate("abowler3.csv","A Bowler3")

34 Future Wickets forecast

import cricpy.analytics as ca
#ca.bowlerPerfForecast("abowler1.csv","A bowler1")

35 Get player data special

import cricpy.analytics as ca
#abowler1sp =ca.getPlayerDataSp(profile1,tdir=".",tfile="abowler1sp.csv",ttype="bowling")
#abowler2sp =ca.getPlayerDataSp(profile2,tdir=".",tfile="abowler2sp.csv",ttype="bowling")
#abowler3sp =ca.getPlayerDataSp(profile3,tdir=".",tfile="abowler3sp.csv",ttype="bowling")

36 Contribution to matches won and lost

Note:This can be done only for Test cricketers

import cricpy.analytics as ca
#ca.bowlerContributionWonLost("abowler1sp.csv","A Bowler1")
#ca.bowlerContributionWonLost("abowler2sp.csv","A Bowler2")
#ca.bowlerContributionWonLost("abowler3sp.csv","A Bowler3")

37 Performance home and overseas

Note:This can be done only for Test cricketers

import cricpy.analytics as ca
#ca.bowlerPerfHomeAway("abowler1sp.csv","A Bowler1")
#ca.bowlerPerfHomeAway("abowler2sp.csv","A Bowler2")
#ca.bowlerPerfHomeAway("abowler3sp.csv","A Bowler3")

38 Relative cumulative average economy rate of bowlers

import cricpy.analytics as ca
frames = ["abowler1.csv","abowler2.csv","abowler3.csv"]
names = ["A Bowler1","A Bowler2","A Bowler3"]
#ca.relativeBowlerCumulativeAvgEconRate(frames,names)

39 Relative Economy Rate against wickets taken

import cricpy.analytics as ca
frames = ["abowler1.csv","abowler2.csv","abowler3.csv"]
names = ["A Bowler1","A Bowler2","A Bowler3"]
#ca.relativeBowlingER(frames,names)

40 Relative cumulative average wickets of bowlers in career

import cricpy.analytics as ca
frames = ["abowler1.csv","abowler2.csv","abowler3.csv"]
names = ["A Bowler1","A Bowler2","A Bowler3"]
#ca.relativeBowlerCumulativeAvgWickets(frames,names)

Clone/download this cricpy template for your own analysis of players. This can be done using RStudio or IPython notebooks from Github at cricpy-template

Key Findings

Analysis of Top 4 batsman

Analysis of Top 3 bowlers

You may also like
1. My book ‘Deep Learning from first principles:Second Edition’ now on Amazon
2. Presentation on ‘Evolution to LTE’
3. Stacks of protocol stacks – A primer
4. Taking baby steps in Lisp
5. Introducing cricket package yorkr: Part 1- Beaten by sheer pace!

To see all posts click Index of posts

My book ‘Deep Learning from first principles:Second Edition’ now on Amazon


The second edition of my book ‘Deep Learning from first principles:Second Edition- In vectorized Python, R and Octave’, is now available on Amazon, in both paperback ($18.99)  and kindle ($9.99/Rs449/-)  versions. Since this book is almost 70% code, all functions, and code snippets have been formatted to use the fixed-width font ‘Lucida Console’. In addition line numbers have been added to all code snippets. This makes the code more organized and much more readable. I have also fixed typos in the book

Untitled

 

The book includes the following chapters

Table of Contents
Preface 4
Introduction 6
1. Logistic Regression as a Neural Network 8
2. Implementing a simple Neural Network 23
3. Building a L- Layer Deep Learning Network 48
4. Deep Learning network with the Softmax 85
5. MNIST classification with Softmax 103
6. Initialization, regularization in Deep Learning 121
7. Gradient Descent Optimization techniques 167
8. Gradient Check in Deep Learning 197
1. Appendix A 214
2. Appendix 1 – Logistic Regression as a Neural Network 220
3. Appendix 2 - Implementing a simple Neural Network 227
4. Appendix 3 - Building a L- Layer Deep Learning Network 240
5. Appendix 4 - Deep Learning network with the Softmax 259
6. Appendix 5 - MNIST classification with Softmax 269
7. Appendix 6 - Initialization, regularization in Deep Learning 302
8. Appendix 7 - Gradient Descent Optimization techniques 344
9. Appendix 8 – Gradient Check 405
References 475

Also see
1. My book ‘Practical Machine Learning in R and Python: Second edition’ on Amazon
2. The 3rd paperback & kindle editions of my books on Cricket, now on Amazon
3. De-blurring revisited with Wiener filter using OpenCV
4. TWS-4: Gossip protocol: Epidemics and rumors to the rescue
5. A Cloud medley with IBM Bluemix, Cloudant DB and Node.js
6. Practical Machine Learning with R and Python – Part 6
7. GooglyPlus: yorkr analyzes IPL players, teams, matches with plots and tables
8. Fun simulation of a Chain in Android

To see posts click Index of Posts