getter

README

EDGAR Getter

EDGAR Getter is a Go package for downloading company periodic reports, filings and forms from Securities and Exchange Commission (SEC) EDGAR site.

Installation

go get -u github.com/RelateTheNews/go-edgar-getter

Usage

package main

import "github.com/RelateTheNews/go-edgar-getter"

func main(){
  var g getter.Getter
  var getURI string
  
  // Note this is a sample URI. Must verify correct URIs on www.sec.gov
  getURI = "https://www.sec.gov/Archives/edgar/Feed/2013/QTR1/"
  
  g.NewGetter()
  
  // files is a list of successfully retrieved files
  files := g.RetrieveURIs(getURI, 0)
  fmt.Println(len(files))
}

Contribution Guidelines

Contributions are greatly appreciated. The maintainers actively manage the issues list. For a list of primary maintainers see ./MAINTAINERS.md. The project follows the typical GitHub pull request model. Before starting any work, please either comment on an existing issue, or file a new one.

Documentation

Overview

Package getter is used to manage retrieval, storge and decompression of Edgar filings from the SEC.

Index

• Constants
• type Getter
  • func (getter *Getter) DownloadableFile(filename string) bool
  • func (getter *Getter) ErrorHandler(e error)
  • func (getter *Getter) NewGetter()
  • func (getter *Getter) RetrieveSingleFile(location string, fileList chan<- string) (string, error)
  • func (getter *Getter) RetrieveURIs(sourceURI string, customLimit int) []string

Constants

const (
	// MAXRETRIEVALSIZE the maximum number of files to retrieve at any one time
	MAXRETRIEVALSIZE = 5000
	// MAXRETRY the maximum number of attempts to download a single resource
	MAXRETRY = 5
)

Types

type Getter

type Getter struct {
	RetrieveURI       string
	SaveLocation      string
	ValidFileSuffixes map[string]bool // ValidFileSuffixes lists the ONLY valid suffixes allowed when downloading
}

Getter provides a data structure to support this package

func (*Getter) DownloadableFile

func (getter *Getter) DownloadableFile(filename string) bool

DownloadableFile checks a file against the approved file suffix list

func (*Getter) ErrorHandler

func (getter *Getter) ErrorHandler(e error)

ErrorHandler a simple error checker and handler which panics if an error exists.

func (*Getter) NewGetter

func (getter *Getter) NewGetter()

NewGetter initializes and performs any setup necessary for the getter package to function.

Items currently Setup:

SaveLocation set to a default value of /tmp/edgar/
ValidFileSuffixes set to default values

func (*Getter) RetrieveSingleFile

func (getter *Getter) RetrieveSingleFile(location string, fileList chan<- string) (string, error)

RetrieveSingleFile retrieves a single file/resource from the specified location. This function can be called concurrently by providing a channel.

func (*Getter) RetrieveURIs

func (getter *Getter) RetrieveURIs(sourceURI string, customLimit int) []string

RetrieveURIs concurrently retrieves all resources from the specified source returning the filenames retrieved.