dplyr do: Some Tips for Using and Programming

Base Concepts of do() (Non Standard Evaluation Version)

As we already said, do() computes arbitrary operations on a data frame returning more than one number back.

To use do(), you must know that:

• it always returns a dataframe
• unlike the others data manipulation verbs of dplyr, do() needs the specification of . placeholder inside the function to apply, referring to the data it has to work with.
  
  R

  # Head of ds ds %>% do(head(.))

  1 2	# Head of ds ds %>% do(head(.))

  R

  Source: local data frame [6 x 3] group x y (fctr) (dbl) (dbl) 1 a 1.995615 -1.71089045 2 a 3.263062 -0.03712943 3 a 2.842166 -0.09022217 4 a 4.773570 0.69742469 5 a 3.233943 2.76536531 6 a 3.637260 4.06379942

  1 2 3 4 5 6 7 8 9 10

  Source: local data frame [6 x 3]      group        x           y   (fctr)    (dbl)       (dbl) 1      a 1.995615 -1.71089045 2      a 3.263062 -0.03712943 3      a 2.842166 -0.09022217 4      a 4.773570  0.69742469 5      a 3.233943  2.76536531 6      a 3.637260  4.06379942

• it is conceived to be used with dplyr group_by() to compute operations within groups:
  
  R

  # Head of ds by group ds %>% group_by(group) %>% do(head(.))

  1 2	# Head of ds by group ds %>% group_by(group) %>% do(head(.))

  R

  Source: local data frame [18 x 3] Groups: group [3] group x y (fctr) (dbl) (dbl) 1 a 1.99561530 -1.71089045 2 a 3.26306233 -0.03712943 3 a 2.84216582 -0.09022217 4 a 4.77356962 0.69742469 5 a 3.23394254 2.76536531 6 a 3.63726018 4.06379942 7 b 2.33415330 -0.56965729 8 b 5.72622741 1.71643653 9 b 2.06170532 4.87756954 10 b 4.68575126 -0.08011508 11 b 0.08401255 -0.04767590 12 b 2.19938816 4.18954758 13 c 3.05634353 -0.89257491 14 c 2.28659319 2.63171152 15 c 4.70525275 1.31450497 16 c 4.02673050 -1.86270620 17 c 5.03640599 2.48564201 18 c 0.95704183 1.27446410

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

  Source: local data frame [18 x 3] Groups: group [3]       group          x           y    (fctr)      (dbl)       (dbl) 1       a 1.99561530 -1.71089045 2       a 3.26306233 -0.03712943 3       a 2.84216582 -0.09022217 4       a 4.77356962  0.69742469 5       a 3.23394254  2.76536531 6       a 3.63726018  4.06379942 7       b 2.33415330 -0.56965729 8       b 5.72622741  1.71643653 9       b 2.06170532  4.87756954 10      b 4.68575126 -0.08011508 11      b 0.08401255 -0.04767590 12      b 2.19938816  4.18954758 13      c 3.05634353 -0.89257491 14      c 2.28659319  2.63171152 15      c 4.70525275  1.31450497 16      c 4.02673050 -1.86270620 17      c 5.03640599  2.48564201 18      c 0.95704183  1.27446410

• the argument of do() can be named or unnamed:
  • named arguments (more than one supplied) become list-columns, with one element for each group:
  R

  # Tail (last 3 obs) of x by group ds %>% group_by(group) %>% do(out=tail(.$x, 3))

  1 2	# Tail (last 3 obs) of x by group ds %>% group_by(group) %>% do(out=tail(.$x, 3))

  R

  Source: local data frame [3 x 2] Groups: group out (fctr) (chr) 1 a <dbl[3]> 2 b <dbl[3]> 3 c <dbl[3]>

  1 2 3 4 5 6 7 8

  Source: local data frame [3 x 2] Groups:      group      out   (fctr)    (chr) 1      a <dbl[3]> 2      b <dbl[3]> 3      c <dbl[3]>

  • unnamed argument (only one supplied) must be a data frame and labels will be duplicated accordingly:
  R

  # Tail (last 3 obs) of x by group ds %>% group_by(group) %>% do(data.frame(out=tail(.$x, 3)))

  1 2	# Tail (last 3 obs) of x by group ds %>% group_by(group) %>% do(data.frame(out=tail(.$x, 3)))

  R

  Source: local data frame [9 x 2] Groups: group [3] group out (fctr) (dbl) 1 a 3.8270397 2 a 0.6426337 3 a 0.6519305 4 b 3.3238824 5 b 0.8290942 6 b 4.1538746 7 c 6.5861213 8 c 4.6280643 9 c 0.3599512

  1 2 3 4 5 6 7 8 9 10 11 12 13 14

  Source: local data frame [9 x 2] Groups: group [3]      group       out   (fctr)     (dbl) 1      a 3.8270397 2      a 0.6426337 3      a 0.6519305 4      b 3.3238824 5      b 0.8290942 6      b 4.1538746 7      c 6.5861213 8      c 4.6280643 9      c 0.3599512

Its use is the same working with customized functions.

Let us define the following function, which performs two simple operations returning a data frame:

If the argument is named the result is:

Otherwise, if argument is unnamed the result is:

Programming with do_() (Standard Evaluation Version)

How can we enclose the previous operations inside a function? Simple! Using do_() (the SE version of do()) and interp() function of lazyeval package.

lazyeval is an R package, written and maintained by Hadley Wickham. It represents a new approach to Non Standard Evaluation (NSE) for R. The difference between SE and NSE approaches is the quoting of input variable names. NSE is suitable for interactive use (see the previous paragraph), but not for programming, for which SE approach is recommended.

Install and load lazyeval, if you haven’t already done it.

interp() helps to build the expression up from a mixture of constants and variables to be passed to .dots argument of dplyr verbs. For more details see Non Standard Evaluation vignette.

In the following example interp() is used to build up the expression to be passed to .dots argument of group_by_() (SE version of group_by()), which consists of the grouping variable name. It is used also to build up the expression to be passed to .dots argument of do_(). This expression consists of the function name specifying also its arguments in brackets.

Let us apply the previous function to ds dataset:

Other Examples

do() is often used to fit models and to display the results.
Look at the following functions!

Let us define a function that fits linear model and returns coefficients as a data frame and apply it to ds by group:

Let us enclose the previous operations inside a function and apply it to ds by group: