System Design Introduction

System Design
Catalogue
  1. 1. Topic
    1. 1.1. Storage
    2. 1.2. Computation
    3. 1.3. Web Application
  2. 2. Data Center
    1. 2.1. Distributed File System
      1. 2.1.1. Hadoop Distributed File system

System/Solution/Product Design

  • Design a Twitter?
  • Design a Uber?
  • Design a short URL service.
    Product -> functionalities/use cases -> Architecture

Topic

Storage

  • Distributed file system
  • Distributed database

    Computation

  • Batch Processing
  • Streaming Processing

    Web Application

Data Center

  • Cluster
  • Rack
  • Node/Server(many computers)

    Distributed File System

  1. Use file system interfaces to manage your data (files and directions)
  2. Data is distributed in many machies
  3. Examples: GFS(Google File System), HDFS, Ceph FS, GlusterFS, MapR FS…
  4. When to use a DFS?
  • Durability

Hadoop Distributed File system

  1. Key features/assumptions
  • Scale up to 100+ PB of storage and a single cluster of several thousand servers,
    supporting close to a billion files and blocks
  1. Designed to run on commodity hardware
  • some components of HDFS is always non-functional

Architecture

Share