wh2o - docs

# Introduction

The following documentation is for those who are looking to contribute to the wh2o project development. This document is broken up by Technology-of-Interest to help you find a part of the project that interests you the most.

See below for a quick overview of the stack. Or skip to the Getting Started section. If you'd like an overview of the project mission, see the About page

# Brief Overview

90% of the codebase is written in Go (opens new window). There are many things about Go that we like, particularly, type safety, built-in testing as well as ease of concurrency and distribution.

The backend is split up into a series of HTTP servers and functions that run in AWS Lambda. Longer running tasks like web scraping and data processing we run on EC2 instances. In an effort to keep costs as minimal as possible, we start and stop the worker EC2 instances on the fly--a scheduled Lambda boots up the instance, the script runs, then when finished the scripts turns itself off via the AWS Go SDK. Spot EC2 instances may be a more cost-effective solution.

# Serverless

As mentioned in the introduction, the majority of the services run as Lambda functions. We use the Serverless Framework (opens new window) to do most of the heavy lifting. In addition to serverless functions, we use a Cockroachdb Serverless Cluster (opens new window) for data.

# Machine Learning

The Gage Reading Forecasting API is written in Python and leverages the Facebook Prophet (opens new window) library and behind a simple Flask (opens new window) HTTP server. The forecasting API also runs serverless.

// @TODO add notes

  • Need to support other non-USGS gage sources
  • Add other inputs/variables like snowpack

# Elasticsearch

In active development for the SearchV2 endpoints, we use Elasticsearch to index gages and rivers.

# Need Help With

  • Query ordering + sorting
  • Pagination strategies

# Browser Automation

We use a series of web scrapers, built with Rod (opens new window), to pull gage readings from various sources. In an effort to be responsible netizens, scrapers have been written in such a way to not overburden target sites/servers.

# React Native

The React Native (RN) application is currently in beta and awaiting approval from the Apple Store. The primary function of the RN client is to send immediate Push notifications in addition to Email and SMS.

# Need Help With

  • Android builds
  • Internationalization

# React + TypeScript

The frontend (opens new window) is a simple SPA bootstrapped with Create React App (opens new window) (we have not had to eject yet) and TypeScript (opens new window).

# Need Help With

  • Responsive behavior. Unfortunately, the frontend was not built mobile-first and we are now playing catch-up.
  • Explore alternatives to antd (opens new window). This framework was chosen because it offers some great components and beautiful micro-animations but out-of-the-box mobile behavior is not so good.

# Internationalization

We currently support English, Spanish and French. The frontend leverages react-i18next (opens new window) for hard coded text content. In the data, each model has an accompanying Translation association for each language.

Development →