Importing and Exporting Data in R

How to read CSV files into R using data.table::fread(), handle common format variations (separators, decimal signs, leading zeros), and save data to CSV, RDS, RData, and Stata formats.

Data Workflow in R Step 1 of 4 Data Preparation with the Tidyverse → View full track →

Packages used in this tutorial:

library(here)
library(data.table)
library(dplyr)

Video lecture

1 Introduction

Functions for importing data take a path to a file, parse that file, and return an R object (usually a kind of data.frame) containing the data.

The most widely used file format for data is .csv — comma separated values. Its big advantage is that it is platform-independent and requires no external software. The downside is that .csv files are relatively large and slower to read than modern binary formats. For most datasets a student will encounter, these disadvantages are irrelevant, making .csv a safe default.

While writing data is usually straightforward, reading data can be frustrating: there are many ways a .csv file can be structured. The package data.table provides data.table::fread(), which handles almost every variation automatically or with minimal configuration. It is also substantially faster than the tidyverse alternative from readr.

NoteWhy not readr?

If you read about data import in tidyverse-focused textbooks, you will find readr functions rather than data.table::fread(). This is one of the few places where breaking dialect consistency is worthwhile: fread() is much faster and more flexible. The only cost is that it returns a data.table rather than a tibble — easily fixed with tibble::as_tibble().

This tutorial assumes you already have an R project set up with a proper directory structure — see the tutorial on Setting Up an R Project if you have not done this yet.

2 Importing CSV data with fread

Assume the dataset you want to import is called exp_data.csv and sits in your project’s data/tidy/ folder. Your project structure looks like this:

Before importing, it is good practice to inspect the raw file in RStudio’s file preview to understand its structure (for large files, use a text editor instead):

The file in our example looks like this:

iso2c,year,exports
AT,2012,53.97
AT,2013,53.44
AT,2014,53.38

This is a standard CSV: comma-separated columns, dot as decimal sign. With default arguments, data.table::fread() reads it correctly — you only need to pass the file path:

here::i_am("R/importing.R")
file_path <- here::here("data/tidy/exp_data.csv")
exp_data <- data.table::fread(file = file_path)
exp_data <- tibble::as_tibble(exp_data)
exp_data

# A tibble: 3 × 3
  iso2c  year exports
  <chr> <int>   <dbl>
1 AT     2012    54.0
2 AT     2013    53.4
3 AT     2014    53.4

The most important optional arguments are:

• sep: column separator
• dec: decimal sign
• colClasses: force column types
• select / drop: include or exclude specific columns
• nrows / skip: limit rows read or skip file headers
• header: whether the first row contains column names

2.1 Separator and decimal sign: sep and dec

In Germany (and many other European countries), ; is used as the column separator and , as the decimal sign. The same data in this format looks like:

iso2c;year;Exporte
AT;2012;53,97
AT;2013;53,44
AT;2014;53,38

fread() often detects non-standard formats automatically, but it is always better to be explicit — it also makes the import faster:

exp_data <- data.table::fread(
  file = file_path,
  sep = ";",
  dec = ","
  )

After any import, inspect the result before proceeding:

exp_data <- tibble::as_tibble(exp_data)
head(exp_data, n = 2)

# A tibble: 2 × 3
  iso2c  year exports
  <chr> <int>   <dbl>
1 AT     2012    54.0
2 AT     2013    53.4

Or use dplyr::glimpse() or str():

str(exp_data)

tibble [3 × 3] (S3: tbl_df/tbl/data.frame)
 $ iso2c  : chr [1:3] "AT" "AT" "AT"
 $ year   : int [1:3] 2012 2013 2014
 $ exports: num [1:3] 54 53.4 53.4

2.2 Column types: colClasses

fread()’s automatic type detection is generally good, but it fails in one important case: numerical codes with leading zeros. Product codes like HS codes are a common example:

commoditycode,complexity
0101,0.06
0102,-0.49
0103,0.51

Without specifying types, fread() reads commoditycode as double, silently stripping the leading zero:

file_path <- here::here("data/tidy/exp_data_hs.csv")
exp_prod_data <- data.table::fread(file = file_path)
tibble::as_tibble(exp_prod_data)

# A tibble: 5 × 2
  commoditycode complexity
          <int>      <dbl>
1           101       0.06
2           102      -0.49
3           103       0.51
4           104      -1.12
5           105      -0.17

Specify the column types via colClasses to preserve leading zeros:

exp_prod_data <- data.table::fread(
  file = file_path,
  colClasses = c("character", "double")
  )
tibble::as_tibble(exp_prod_data)

# A tibble: 5 × 2
  commoditycode complexity
  <chr>              <dbl>
1 0101                0.06
2 0102               -0.49
3 0103                0.51
4 0104               -1.12
5 0105               -0.17

For data with many columns, rep() saves typing — e.g. rep("double", 6) for six consecutive double columns.

2.3 Reading a subset of rows: nrows and skip

Specifying column types requires knowing the column structure. For large files you cannot open in a text editor, read a small number of rows first:

exp_data <- tibble::as_tibble(data.table::fread(
  file = here::here("data/tidy/exp_data.csv"),
  nrows = 1)
  )
exp_data

# A tibble: 1 × 3
  iso2c  year exports
  <chr> <int>   <dbl>
1 AT     2012    54.0

Some files contain a text header above the actual data:

This is awesome data from 2012-2014
It was compiled by Claudius
He also added this useless header
iso2c,year,Exporte
AT,2012,53.97

Reading without skip produces garbled output:

exp_data <- data.table::fread(
  file = here::here("data/tidy/exp_data_header.csv")
  )
tibble::as_tibble(exp_data)

# A tibble: 1 × 6
  V1    It    was   compiled be      Claudius
  <chr> <chr> <chr> <chr>    <chr>   <chr>   
1 He    also  added this     useless header  

Pass skip = 3 to ignore the first three rows:

exp_data <- tibble::as_tibble(data.table::fread(
  file = here::here("data/tidy/exp_data_header.csv"),
  skip = 3)
  )
exp_data

# A tibble: 3 × 3
  iso2c  year Exporte
  <chr> <int>   <dbl>
1 AT     2012    54.0
2 AT     2013    53.4
3 AT     2014    53.4

2.4 Selecting columns: select and drop

To read only a subset of columns — useful for speed with large files:

exp_data <- data.table::fread(
  file = here::here("data/tidy/exp_data.csv"),
  nrows = 1,
  select = c("year", "exports")
  )
exp_data <- tibble::as_tibble(exp_data)
exp_data

# A tibble: 2 × 2
   year exports
  <int>   <dbl>
1  2012    54.0
2  2013    53.4

You can combine select with type specification using a named vector:

exp_data <- data.table::fread(
  file = here::here("data/tidy/exp_data.csv"),
  nrows = 1,
  select = c("year" = "double", "exports" = "double")
  )
exp_data <- tibble::as_tibble(exp_data)
exp_data

# A tibble: 2 × 2
   year exports
  <dbl>   <dbl>
1  2012    54.0
2  2013    53.4

Alternatively, drop specifies columns to exclude:

exp_data <- data.table::fread(
  file = here::here("data/tidy/exp_data.csv"),
  nrows = 1,
  drop = "iso2c"
  )
exp_data <- tibble::as_tibble(exp_data)
exp_data

# A tibble: 2 × 2
   year exports
  <int>   <dbl>
1  2012    54.0
2  2013    53.4

3 Importing other file formats

3.1 R-native formats: .rds and .RData

R has two proprietary formats that are fast, compressed, and platform-compatible — but can only be opened with R.

.rds stores a single object. Read it with readRDS():

data_set <- readRDS(file = here::here("data/tidy/exp_data.Rds"))

.RData can store multiple objects. Read with load() — note that the objects are placed directly into your environment with their original names, without any assignment:

load(here::here("data/tidy/test_data.RData"))

test_dat

  a b
1 1 3
2 2 4

test_vec

[1] "Test vector"

3.2 Stata files: .dta

The haven package reads Stata’s native format:

dta_data <- haven::read_dta(file = here::here("data/tidy/exp_data.dta"))

haven also supports SAS and SPSS formats via similarly named functions.

4 Exporting data

4.1 CSV with fwrite

data.table::fwrite() is the fastest function for writing CSV files and the natural companion to fread():

data.table::fwrite(
  x = test_data,
  file = here::here("data/tidy/test_data.csv")
)

CSV is the right default for sharing data: it is platform-independent and readable by any software.

4.2 R-native formats

For single objects, saveRDS() with optional compression:

saveRDS(
  object = test_data,
  file = here::here("data/tidy/test_data.Rds"),
  compress = "gz"   # gz = fast, bz2 = smallest, xz = middle ground
)

For multiple objects at once, save():

save(
  list = c("test_data", "another_object"),
  file = here::here("data/tidy/datacollection.RData")
)

Use R-native formats when all consumers of the data use R — they are faster and smaller than CSV for large datasets.

4.3 Stata format

haven::write_dta(
  data = test_data,
  path = here::here("data/tidy/test_data.dta")
)

5 Practice

The ImportData exercise set in the DataScienceExercises package covers the main concepts from this tutorial:

learnr::run_tutorial(
  name = "ImportData",
  package = "DataScienceExercises",
  shiny_args = list("launch.browser" = TRUE)
)

See the blog post on DataScienceExercises for installation instructions.

6 Further reading

• Efficient R Input/Output — benchmark comparison of file formats and read/write functions
• R data frame benchmark — detailed speed comparison
• help(fread) — the full reference for all fread() arguments