Advanced Big Data Analysis and Management Using R - online

This two-day online course provides advanced training in Big Data Analysis and Management using R, focusing on efficient techniques for processing, managing, and visualizing large datasets. Participants will learn to overcome common challenges in big data workflows, including memory optimization, time series analysis, and geospatial data handling. The course combines theory with hands-on practice, equipping learners with practical skills for real-world data applications.

The course covers: 

Introduction to Big Data and R Environment

1. Big data concepts
2. Challenges of handling Big Data (Memory limits, computational efficiency)
3. R/RStudio setup, Package installations

Handling Large Datasets

1. Working with large datasets using data.table
2. Memory-efficient data wrangling with dplyr
3. Working with Out-of-Memory data (disk.frame, ff, feather)
4. Best practices for efficient pipelines

Visualization in Big Data Context

1. Challenges of visualizing big data
2. Exploratory data analysis with ggplot2 and plotly
3. Handling large datasets in visualizations (sampling, aggregation, ggforce)
4. Overview of Shiny dashboard - example

Handling Time Series (Temporal) Data

1. Temporal data structures in R (Date, lubridate)
2. Time series storage and manipulation (xts, zoo)
3. Time series aggregation and decomposition
4. Temporal visualization techniques (ggfortify, gghighlight)
5. Interactive time exploration with dygraphs

Handling Geospatial Data

1. Geospatial vector data structure in R (sf, sp)
2. Handling Raster data in R (terra, raster)
3. Projections and CRS: understanding EPSG codes and proj4 strings
4. Static Maps: ggplot2 with geom_sf, tmap for thematic mapping
5. Interactive Maps: leaflet, mapview, and plotly integration

By the end of the course participants will:

• Understand key concepts and challenges of big data analysis in R.
• Efficiently handle and process large datasets using optimized techniques.
• Apply effective visualization methods for exploratory data analysis.
• Manage and analyse time series data.
• Work with geospatial data for mapping and spatial analysis.
• Build streamlined workflows for scalable big data solutions.