R basics

Setup

Other than data.frame, packages need to be installed before getting to this stage.

data.frame

No setup needed: this is part of Base R.

tibble

Either library(tibble) or (preferably) library(tidyverse). The latter also brings in important libraries such as dplyr.

library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.2.1     ✔ readr     2.2.0
✔ forcats   1.0.1     ✔ stringr   1.6.0
✔ ggplot2   4.0.3     ✔ tibble    3.3.1
✔ lubridate 1.9.5     ✔ tidyr     1.3.2
✔ purrr     1.2.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

Documentation is fairly extensive, in the Tidyverse style:

A website.
A function reference.
A chapter in R for Data Science.

data.table

Caution: this does not always play nicely with tidyverse. It is designed as an alternative approach, not a complement.

library(data.table)


Attaching package: 'data.table'

The following objects are masked from 'package:lubridate':

    hour, isoweek, isoyear, mday, minute, month, quarter, second, wday,
    week, yday, year

The following objects are masked from 'package:dplyr':

    between, first, last

The following object is masked from 'package:purrr':

    transpose

The following object is masked from 'package:base':

    %notin%

The first time the library is attached, it gives configuration information. On my system:

data.table 1.18.4 using 8 threads (see ?getDTthreads).  Latest news: r-datatable.com

Polars

The R bindings for Polars are available in r-multiverse, but not in CRAN.

DuckDB

library("duckdb")

Loading required package: DBI

Arrow

TODO

PostgreSQL

TODO

Creating

data.frame

# create the column vectors
languages <- c("Fortran", "R", "Python", "Julia")
created <- c(1957, 1993, 1991, 2012)
has.syllabus <- c(FALSE, TRUE, TRUE, TRUE)

# join columns to create the dataframe
df <- data.frame(languages, created, has.syllabus)
df

  languages created has.syllabus
1   Fortran    1957        FALSE
2         R    1993         TRUE
3    Python    1991         TRUE
4     Julia    2012         TRUE

# look at the structure
str(df)

'data.frame':   4 obs. of  3 variables:
 $ languages   : chr  "Fortran" "R" "Python" "Julia"
 $ created     : num  1957 1993 1991 2012
 $ has.syllabus: logi  FALSE TRUE TRUE TRUE

tibble

# column vectors are same as for data.frame
tbl <- tibble(languages, created, has.syllabus)
tbl

# A tibble: 4 × 3
  languages created has.syllabus
  <chr>       <dbl> <lgl>       
1 Fortran      1957 FALSE       
2 R            1993 TRUE        
3 Python       1991 TRUE        
4 Julia        2012 TRUE

str(tbl)

tibble [4 × 3] (S3: tbl_df/tbl/data.frame)
 $ languages   : chr [1:4] "Fortran" "R" "Python" "Julia"
 $ created     : num [1:4] 1957 1993 1991 2012
 $ has.syllabus: logi [1:4] FALSE TRUE TRUE TRUE

data.table

Polars

DuckDB

Arrow

Importing data

data.frame

tibble

data.table

Polars

DuckDB

Arrow

Subsetting

data.frame & tibble

Dataframes, including tibbles, can be treated as lists of column vectors, so list indexing recovers a specified column.

tbl

# A tibble: 4 × 3
  languages created has.syllabus
  <chr>       <dbl> <lgl>       
1 Fortran      1957 FALSE       
2 R            1993 TRUE        
3 Python       1991 TRUE        
4 Julia        2012 TRUE

tbl$created

[1] 1957 1993 1991 2012

A dataframe can also be indexed with matrix-style indexing.

tbl[c(2, 4), 1:2]

# A tibble: 2 × 2
  languages created
  <chr>       <dbl>
1 R            1993
2 Julia        2012

Using dplyr, get a single column with pull() with the name or sequential number (negative numbers to count right-to-left).

tbl |> pull(created)

[1] 1957 1993 1991 2012

This is the same result as tbl$created, but using a pipeline-friendly function.

To get multiple columns, the appropriate function is select(), which is highly versatile. Get (or drop) columns based on properties of their name or type.

# Range with position and/or name
tbl |> select(1:created)

# A tibble: 4 × 2
  languages created
  <chr>       <dbl>
1 Fortran      1957
2 R            1993
3 Python       1991
4 Julia        2012

# Exclude a column
tbl |> select(!created)

# A tibble: 4 × 2
  languages has.syllabus
  <chr>     <lgl>       
1 Fortran   FALSE       
2 R         TRUE        
3 Python    TRUE        
4 Julia     TRUE

# Use type of column
tbl |> select(where(is.numeric))

# A tibble: 4 × 1
  created
    <dbl>
1    1957
2    1993
3    1991
4    2012

Multiple criteria are allowed, using Boolean operators &, | and ! (and, or not).

Column names that are valid R identifiers do not need quotes within a select(). Invalid names (e.g. those including spaces) can be enclosed in backticks, though renaming them might be better.

The select() function can work with a range of helper functions to pick column names: starts_with, contains, num_range and various others. matches allows full RegEx matching. See the documentation for details.

# limit display to top 3 rows of non-list columns
starwars |> 
  select(!where(is.list)) |> 
  head(3)

# A tibble: 3 × 11
  name      height  mass hair_color skin_color eye_color birth_year sex   gender
  <chr>      <int> <dbl> <chr>      <chr>      <chr>          <dbl> <chr> <chr> 
1 Luke Sky…    172    77 blond      fair       blue              19 male  mascu…
2 C-3PO        167    75 <NA>       gold       yellow           112 none  mascu…
3 R2-D2         96    32 <NA>       white, bl… red               33 none  mascu…
# ℹ 2 more variables: homeworld <chr>, species <chr>

# pick a subset of columns
starwars |> 
  select(name | ends_with("color")) |> 
  head(5)

# A tibble: 5 × 4
  name           hair_color skin_color  eye_color
  <chr>          <chr>      <chr>       <chr>    
1 Luke Skywalker blond      fair        blue     
2 C-3PO          <NA>       gold        yellow   
3 R2-D2          <NA>       white, blue red      
4 Darth Vader    none       white       yellow   
5 Leia Organa    brown      light       brown

Get rows matching some criteria with filter(), or exclude them with filter_out().

starwars |> 
  select(name:mass) |> 
  filter(between(height, 150, 165) & !is.na(mass))

# A tibble: 4 × 3
  name               height  mass
  <chr>               <int> <dbl>
1 Leia Organa           150    49
2 Beru Whitesun Lars    165    75
3 Nien Nunb             160    68
4 Ben Quadinaros        163    65

Filter criteria can be arbitrarily complex, but always based on row contents.

If row numbers are known, we can use a variety of slice() functions.

starwars |> 
  select(name | homeworld) |> 
  slice(20:25)

# A tibble: 6 × 2
  name             homeworld
  <chr>            <chr>    
1 Palpatine        Naboo    
2 Boba Fett        Kamino   
3 IG-88            <NA>     
4 Bossk            Trandosha
5 Lando Calrissian Socorro  
6 Lobot            Bespin

# random sample of rows
starwars |> 
  select(name | homeworld) |> 
  slice_sample(n = 4)

# A tibble: 4 × 2
  name           homeworld
  <chr>          <chr>    
1 Jocasta Nu     Coruscant
2 Roos Tarpals   Naboo    
3 Barriss Offee  Mirial   
4 Wedge Antilles Corellia

To remove duplicate rows, use distinct().

data.table

TODO

Polars

TODO

DuckDB

TODO

Arrow

TODO

Sorting

data.frame & tibble

arrange() sorts rows by the values in one or more columns.

tbl

# A tibble: 4 × 3
  languages created has.syllabus
  <chr>       <dbl> <lgl>       
1 Fortran      1957 FALSE       
2 R            1993 TRUE        
3 Python       1991 TRUE        
4 Julia        2012 TRUE

tbl |> arrange(languages)

# A tibble: 4 × 3
  languages created has.syllabus
  <chr>       <dbl> <lgl>       
1 Fortran      1957 FALSE       
2 Julia        2012 TRUE        
3 Python       1991 TRUE        
4 R            1993 TRUE

data.table

Polars

TODO

DuckDB

TODO

Arrow

TODO

Modifying

data.frame & tibble

Column names can be changed with rename(newname = oldname), or rename_with() to apply a function. A typical use would be cleaning up imported names to make them easier to work with in R, by removing whitespace and forcing a consistent format for related names.

Note the syntax within rename(). The contents of column oldname are bound to name newname, hence the order.

Column order can be changed with relocate(). Specified column(s) are moved to the left-most position(s) by default, but a .before or .after argument can be used for finer positioning.

sw <- starwars |> select(name:species) |> slice(1:4)
sw

# A tibble: 4 × 11
  name      height  mass hair_color skin_color eye_color birth_year sex   gender
  <chr>      <int> <dbl> <chr>      <chr>      <chr>          <dbl> <chr> <chr> 
1 Luke Sky…    172    77 blond      fair       blue            19   male  mascu…
2 C-3PO        167    75 <NA>       gold       yellow         112   none  mascu…
3 R2-D2         96    32 <NA>       white, bl… red             33   none  mascu…
4 Darth Va…    202   136 none       white      yellow          41.9 male  mascu…
# ℹ 2 more variables: homeworld <chr>, species <chr>

sw |> relocate(c(species, homeworld), .after = name)

# A tibble: 4 × 11
  name           species homeworld height  mass hair_color skin_color  eye_color
  <chr>          <chr>   <chr>      <int> <dbl> <chr>      <chr>       <chr>    
1 Luke Skywalker Human   Tatooine     172    77 blond      fair        blue     
2 C-3PO          Droid   Tatooine     167    75 <NA>       gold        yellow   
3 R2-D2          Droid   Naboo         96    32 <NA>       white, blue red      
4 Darth Vader    Human   Tatooine     202   136 none       white       yellow   
# ℹ 3 more variables: birth_year <dbl>, sex <chr>, gender <chr>

For bigger changes, mutate() lets you:

Create new columns that are functions of existing columns.
Replace an existing column, by creating a new column with the same name.
Delete a column, by setting its value to NULL

starwars |> 
  select(c(name, species, height, mass)) |> 
  mutate(BMI = mass / (height / 100)^2) |> 
  head(4)

# A tibble: 4 × 5
  name           species height  mass   BMI
  <chr>          <chr>    <int> <dbl> <dbl>
1 Luke Skywalker Human      172    77  26.0
2 C-3PO          Droid      167    75  26.9
3 R2-D2          Droid       96    32  34.7
4 Darth Vader    Human      202   136  33.3

When you want to operate on a subset of the columns with functions such as mutate(), the select() |> mutate() sequence in the above example is one option. Only the selected columns will be in the result.

Alternatively, it can be convenient to use pick() within the mutate() call:

starwars |> mutate(pick(c(name, species, height, mass)), BMI = mass / (height / 100)^2) |> head(4)

# A tibble: 4 × 15
  name      height  mass hair_color skin_color eye_color birth_year sex   gender
  <chr>      <int> <dbl> <chr>      <chr>      <chr>          <dbl> <chr> <chr> 
1 Luke Sky…    172    77 blond      fair       blue            19   male  mascu…
2 C-3PO        167    75 <NA>       gold       yellow         112   none  mascu…
3 R2-D2         96    32 <NA>       white, bl… red             33   none  mascu…
4 Darth Va…    202   136 none       white      yellow          41.9 male  mascu…
# ℹ 6 more variables: homeworld <chr>, species <chr>, films <list>,
#   vehicles <list>, starships <list>, BMI <dbl>

Alternatively, it can be convenient to use pick() within the mutate() call:

starwars |> mutate(pick(c(name, species, height, mass)), BMI = mass / (height / 100)^2) |> head(4)

# A tibble: 4 × 15
  name      height  mass hair_color skin_color eye_color birth_year sex   gender
  <chr>      <int> <dbl> <chr>      <chr>      <chr>          <dbl> <chr> <chr> 
1 Luke Sky…    172    77 blond      fair       blue            19   male  mascu…
2 C-3PO        167    75 <NA>       gold       yellow         112   none  mascu…
3 R2-D2         96    32 <NA>       white, bl… red             33   none  mascu…
4 Darth Va…    202   136 none       white      yellow          41.9 male  mascu…
# ℹ 6 more variables: homeworld <chr>, species <chr>, films <list>,
#   vehicles <list>, starships <list>, BMI <dbl>

Only the picked columns are used in the mutation, but all columns are returned.

data.table

Polars

TODO

DuckDB

TODO

Arrow

TODO