exporter

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 14, 2025 License: Apache-2.0 Imports: 30 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DISCARD   = "DISCARD"   // Ignore this column (when SELECT *)
	LABEL     = "LABEL"     // Use this column as a label
	COUNTER   = "COUNTER"   // Use this column as a counter
	GAUGE     = "GAUGE"     // Use this column as a gauge
	HISTOGRAM = "HISTOGRAM" // Use this column as a histogram
)

Variables

View Source 
var (
	PgExporter *Exporter
	ReloadLock sync.Mutex
	Logger     *slog.Logger
)

PgExporter is the global singleton of Exporter

View Source 
var ColumnUsage = map[string]bool{
	DISCARD:   false,
	LABEL:     false,
	COUNTER:   true,
	GAUGE:     true,
	HISTOGRAM: true,
}

ColumnUsage determine how to use query result column

View Source 
var Version = "0.8.0"

Version is read by make build procedure

Functions

func DryRun 

func DryRun()

DryRun will explain all query fetched from configs

func DummyServer 

func DummyServer() (s *http.Server, exit <-chan bool)

DummyServer response with a dummy metrics pg_up 0 or pgbouncer_up 0

func GetConfig 

func GetConfig() (res string)

GetConfig will try load config from target path

func GetPGURL 

func GetPGURL() string

GetPGURL will retrieve, parse, modify postgres connection string

func LoadConfig 

func LoadConfig(configPath string) (queries map[string]*Query, err error)

LoadConfig will read single conf file or read multiple conf file if a dir is given conf file in a dir will be load in alphabetic order, query with same name will overwrite predecessor

func ParseArgs 

func ParseArgs()

ParseArgs will parse cli args with kingpin. url and config have special treatment

func ParseConfig 

func ParseConfig(content []byte) (queries map[string]*Query, err error)

ParseConfig turn config content into Query struct

func ParseDatname 

func ParseDatname(pgurl string) string

ParseDatname extract database name part of a pgurl

func ParseSemver 

func ParseSemver(semverStr string) int

ParseSemver will turn semantic version string into integer

func PgbouncerPrecheck 

func PgbouncerPrecheck(s *Server) (err error)

PgbouncerPrecheck checks pgbouncer connection before scrape

func PostgresPrecheck 

func PostgresPrecheck(s *Server) (err error)

PostgresPrecheck checks postgres connection and gathering facts if any important fact changed, it will trigger a plan before next scrape

func ProcessPGURL 

func ProcessPGURL(pgurl string) string

ProcessPGURL will fix URL with default options

func Reload 

func Reload() error

Reload will launch a new pg exporter instance

func ReloadFunc 

func ReloadFunc(w http.ResponseWriter, r *http.Request)

ReloadFunc handles reload request

func ReplaceDatname 

func ReplaceDatname(pgurl, datname string) string

ReplaceDatname will replace pgurl with new database name

func RetrievePGURL 

func RetrievePGURL() (res string)

RetrievePGURL retrieve pg target url from multiple sources according to precedence priority: cli-args > env > env file path

  1. Command Line Argument (--url -u -d)
  2. Environment PG_EXPORTER_URL
  3. From file specified via Environment PG_EXPORTER_URL_FILE
  4. Default url

func Run 

func Run()

Run pg_exporter

func ShadowPGURL 

func ShadowPGURL(pgurl string) string

ShadowPGURL will hide password part of dsn

func TitleFunc 

func TitleFunc(w http.ResponseWriter, r *http.Request)

TitleFunc responding a description message

func VersionFunc 

func VersionFunc(w http.ResponseWriter, r *http.Request)

VersionFunc responding current pg_exporter version

Types

type Collector 

type Collector struct {
	*Query
	Server *Server // It's a query, but holds a server
	// contains filtered or unexported fields
}

Collector holds runtime information of a Query running on a Server It is deeply coupled with Server. Besides, it can be a collector itself

func NewCollector 

func NewCollector(q *Query, s *Server) *Collector

NewCollector will generate query instance from query, Injecting a server object

func (*Collector) CacheHit 

func (q *Collector) CacheHit() bool

CacheHit report whether last scrape was serve from cache

func (*Collector) Collect 

func (q *Collector) Collect(ch chan<- prometheus.Metric)

Collect implement prometheus.Collector

func (*Collector) Describe 

func (q *Collector) Describe(ch chan<- *prometheus.Desc)

Describe implement prometheus.Collector

func (*Collector) Duration 

func (q *Collector) Duration() float64

Duration returns last scrape duration in float64 seconds

func (*Collector) Error 

func (q *Collector) Error() error

Error wraps query error (including error in predicate query)

func (*Collector) PredicateSkip added in v0.7.0 

func (q *Collector) PredicateSkip() (bool, string)

PredicateSkip tells the last scrape skip due to predicate query and if so which predicate query caused the skip?

func (*Collector) ResultSize 

func (q *Collector) ResultSize() int

ResultSize report last scraped metric count

type Column 

type Column struct {
	Name    string    `yaml:"name"`
	Usage   string    `yaml:"usage,omitempty"`   // column usage
	Rename  string    `yaml:"rename,omitempty"`  // rename column
	Bucket  []float64 `yaml:"bucket,omitempty"`  // histogram bucket
	Scale   string    `yaml:"scale,omitempty"`   // scale factor
	Default string    `yaml:"default,omitempty"` // default value
	Desc    string    `yaml:"description,omitempty"`
}

Column holds the metadata of query result

func (*Column) MetricDesc 

func (c *Column) MetricDesc(prefix string, labels []string) *MetricDesc

MetricDesc will generate MetricDesc from column and additional information

func (*Column) PrometheusValueType 

func (c *Column) PrometheusValueType() prometheus.ValueType

PrometheusValueType returns column's corresponding prometheus value type

func (*Column) String 

func (c *Column) String() string

String turns column into a one-line text representation

type Exporter 

type Exporter struct {
	// contains filtered or unexported fields
}

Exporter implement prometheus.Collector interface exporter contains one or more (auto-discover-database) servers that can scrape metrics with Query

func NewExporter 

func NewExporter(dsn string, opts ...ExporterOpt) (e *Exporter, err error)

NewExporter construct a PG Exporter instance for given dsn

func (*Exporter) Check 

func (e *Exporter) Check()

Check will perform an immediate server health check

func (*Exporter) Close 

func (e *Exporter) Close()

Close will close all underlying servers

func (*Exporter) Collect 

func (e *Exporter) Collect(ch chan<- prometheus.Metric)

Collect implement prometheus.Collector

func (*Exporter) CreateServer 

func (e *Exporter) CreateServer(dbname string)

CreateServer will spawn new database server from a database name combined with existing dsn string This happens when a database is newly created

func (*Exporter) Describe 

func (e *Exporter) Describe(ch chan<- *prometheus.Desc)

Describe implement prometheus.Collector

func (*Exporter) Explain 

func (e *Exporter) Explain() string

Explain is just yet another wrapper of server.ExplainHTML

func (*Exporter) ExplainFunc 

func (e *Exporter) ExplainFunc(w http.ResponseWriter, r *http.Request)

ExplainFunc expose explain document

func (*Exporter) IterateServer 

func (e *Exporter) IterateServer() (res []*Server)

IterateServer will get snapshot of extra servers

func (*Exporter) OnDatabaseChange 

func (e *Exporter) OnDatabaseChange(change map[string]bool)

OnDatabaseChange will spawn new Server when new database is created and destroy Server if corresponding database is dropped

func (*Exporter) PrimaryCheckFunc 

func (e *Exporter) PrimaryCheckFunc(w http.ResponseWriter, r *http.Request)

PrimaryCheckFunc tells whether target instance is a primary, 200 yes 404 no 503 unknown

func (*Exporter) Recovery 

func (e *Exporter) Recovery() bool

Recovery will delegate primary/replica check to primary server

func (*Exporter) RemoveServer 

func (e *Exporter) RemoveServer(dbname string)

RemoveServer will destroy Server from Exporter according to database name This happens when a database is dropped

func (*Exporter) ReplicaCheckFunc 

func (e *Exporter) ReplicaCheckFunc(w http.ResponseWriter, r *http.Request)

ReplicaCheckFunc tells whether target instance is a replica, 200 yes 404 no 503 unknown

func (*Exporter) Stat 

func (e *Exporter) Stat() string

Stat is just yet another wrapper of server.Stat

func (*Exporter) StatFunc 

func (e *Exporter) StatFunc(w http.ResponseWriter, r *http.Request)

StatFunc expose html statistics

func (*Exporter) Status 

func (e *Exporter) Status() string

Status will report 4 available status: primary|replica|down|unknown

func (*Exporter) Up 

func (e *Exporter) Up() bool

Up will delegate aliveness check to primary server

func (*Exporter) UpCheckFunc 

func (e *Exporter) UpCheckFunc(w http.ResponseWriter, r *http.Request)

UpCheckFunc tells whether target instance is alive, 200 up 503 down

type ExporterOpt 

type ExporterOpt func(*Exporter)

ExporterOpt configures Exporter

func WithAutoDiscovery 

func WithAutoDiscovery(flag bool) ExporterOpt

WithAutoDiscovery configures exporter with excluded database

func WithCacheDisabled 

func WithCacheDisabled(disableCache bool) ExporterOpt

WithCacheDisabled set cache param to exporter

func WithConfig 

func WithConfig(configPath string) ExporterOpt

WithConfig add config path to Exporter

func WithConfigReader added in v0.7.1 

func WithConfigReader(reader io.Reader) ExporterOpt

WithConfigReader uses a the provided reader to load a configuration for the Exporter

func WithConnectTimeout 

func WithConnectTimeout(timeout int) ExporterOpt

WithConnectTimeout will specify timeout for conn pre-check. It's useful to increase this value when monitoring a remote instance (cross DC, cross AZ)

func WithConstLabels 

func WithConstLabels(s string) ExporterOpt

WithConstLabels add const label to exporter. 0 length label returns nil

func WithExcludeDatabase 

func WithExcludeDatabase(excludeStr string) ExporterOpt

WithExcludeDatabase configures exporter with excluded database

func WithFailFast 

func WithFailFast(failFast bool) ExporterOpt

WithFailFast marks exporter fail instead of waiting during start-up

func WithIncludeDatabase 

func WithIncludeDatabase(includeStr string) ExporterOpt

WithIncludeDatabase configures exporter with included database

func WithIntroDisabled 

func WithIntroDisabled(disableIntro bool) ExporterOpt

WithIntroDisabled will pass introspection option to server

func WithNamespace 

func WithNamespace(namespace string) ExporterOpt

WithNamespace will specify metric namespace, by default is pg or pgbouncer

func WithTags 

func WithTags(tags string) ExporterOpt

WithTags will register given tags to Exporter and all belonged servers

type MetricDesc 

type MetricDesc struct {
	Name   string
	Labels []string
	Column *Column
}

MetricDesc is generated by collector's column definition

func (*MetricDesc) String 

func (m *MetricDesc) String() string

Signature will print metric signature such as pg_db_age{datname}

type PredicateQuery added in v0.7.0 

type PredicateQuery struct {
	Name string  `yaml:"name,omitempty"`  // predicate query name, only used for logging
	SQL  string  `yaml:"predicate_query"` // SQL command to return a predicate
	TTL  float64 `yaml:"ttl,omitempty"`   // How long to cache results for
}

A PredicateQuery is a query that returns a 1-column resultset that's used to decide whether to run the main query.

type Query 

type Query struct {
	Name             string           `yaml:"name,omitempty"`              // actual query name, used as metric prefix
	Desc             string           `yaml:"desc,omitempty"`              // description of this metric query
	SQL              string           `yaml:"query"`                       // SQL command to fetch metrics
	PredicateQueries []PredicateQuery `yaml:"predicate_queries,omitempty"` // SQL command to filter metrics
	Branch           string           `yaml:"-"`                           // branch name, top layer key of config file

	// control query behaviour
	Tags       []string `yaml:"tags,omitempty"`        // tags are used for execution control
	TTL        float64  `yaml:"ttl,omitempty"`         // caching ttl in seconds
	Timeout    float64  `yaml:"timeout,omitempty"`     // query execution timeout in seconds
	Priority   int      `yaml:"priority,omitempty"`    // execution priority, from 1 to 999
	MinVersion int      `yaml:"min_version,omitempty"` // minimal supported version, include
	MaxVersion int      `yaml:"max_version,omitempty"` // maximal supported version, not include
	Fatal      bool     `yaml:"fatal,omitempty"`       // if query marked fatal fail, entire scrape will fail
	Skip       bool     `yaml:"skip,omitempty"`        // if query marked skip, it will be omit while loading

	Metrics []map[string]*Column `yaml:"metrics"` // metric definition list

	// metrics parsing auxiliaries
	Path        string             `yaml:"-"` // where am I from ?
	Columns     map[string]*Column `yaml:"-"` // column map
	ColumnNames []string           `yaml:"-"` // column names in origin orders
	LabelNames  []string           `yaml:"-"` // column (name) that used as label, sequences matters
	MetricNames []string           `yaml:"-"` // column (name) that used as metric
}

Query hold the information of how to fetch metric and parse them

func ParseQuery 

func ParseQuery(config string) (*Query, error)

ParseQuery generate a single query from config string

func (*Query) ColumnList 

func (q *Query) ColumnList() (res []*Column)

ColumnList return ordered column list

func (*Query) Explain 

func (q *Query) Explain() string

Explain will turn query into text format

func (*Query) HTML 

func (q *Query) HTML() string

HTML will turn Query into HTML format

func (*Query) HasTag 

func (q *Query) HasTag(tag string) bool

HasTag tells whether this query have specific tag since only few tags is provided, we don't really need a map here

func (*Query) LabelList 

func (q *Query) LabelList() []string

LabelList returns a list of label column names

func (*Query) MarshalYAML 

func (q *Query) MarshalYAML() string

MarshalYAML will turn query into YAML format

func (*Query) MetricList 

func (q *Query) MetricList() (res []*MetricDesc)

MetricList returns a list of MetricDesc generated by this query

func (*Query) TimeoutDuration 

func (q *Query) TimeoutDuration() time.Duration

TimeoutDuration will turn timeout settings into time.Duration

type Server 

type Server struct {
	*sql.DB // database instance (do not close this due to the stupid implementation in database/sql)

	// postgres fact gather from server
	UP       bool   // indicate whether target server is connectable
	Recovery bool   // is server in recovering
	Version  int    // pg server version num
	Database string // database name of current server connection
	Username string // current username

	Databases map[string]bool // all available database in target cluster

	Namespaces map[string]bool // all available schema in target cluster
	Extensions map[string]bool // all available extension in target cluster

	Tags            []string // server tags set by cli arg --tag
	PgbouncerMode   bool     // indicate it is a pgbouncer server
	DisableCache    bool     // force executing, ignoring caching policy
	ExcludeDbnames  []string // if ExcludeDbnames is provided, Auto Database Discovery is enabled
	Forked          bool     // is this a forked server ? (does not run cluster level query)
	Planned         bool     // if false, server will trigger a plan before collect
	ConnectTimeout  int      // connect timeout for this server in ms
	ConnMaxLifetime int      // connection max lifetime for this server in seconds

	// query
	Collectors []*Collector // query collector instance (installed query)
	// contains filtered or unexported fields
}

Server represent a postgres connection, with additional fact, conf, runtime info

func NewServer 

func NewServer(dsn string, opts ...ServerOpt) *Server

NewServer will check dsn, but not trying to connect

func (*Server) Check 

func (s *Server) Check() error

Check will issue a connection and executing precheck hook function

func (*Server) Collect 

func (s *Server) Collect(ch chan<- prometheus.Metric)

Collect implement prometheus.Collector interface

func (*Server) Compatible 

func (s *Server) Compatible(query *Query) (res bool, reason string)

Compatible tells whether a query is compatible with current server

func (*Server) Describe 

func (s *Server) Describe(ch chan<- *prometheus.Desc)

Describe implement prometheus.Collector

func (*Server) Duration 

func (s *Server) Duration() float64

Duration returns last scrape duration in float64 seconds

func (*Server) Error 

func (s *Server) Error() error

func (*Server) Explain 

func (s *Server) Explain() string

Explain will print all queries that registered to server

func (*Server) ExplainHTML 

func (s *Server) ExplainHTML() string

ExplainHTML will print server stats in HTML format

func (*Server) GetConnectTimeout 

func (s *Server) GetConnectTimeout() time.Duration

func (*Server) HasTag 

func (s *Server) HasTag(tag string) bool

HasTag tells whether this server have specific tag

func (*Server) Name 

func (s *Server) Name() string

Name is coalesce(s.Database, dsn)

func (*Server) Plan 

func (s *Server) Plan(queries ...*Query)

Plan will install queries that compatible with server fact (version, level, recovery, plugin, tags,...)

func (*Server) ResetStats 

func (s *Server) ResetStats()

ResetStats will clear all statistic info

func (*Server) Stat 

func (s *Server) Stat() string

Stat will turn Server internal stats into HTML

func (*Server) Uptime 

func (s *Server) Uptime() float64

Uptime returns servers's uptime

type ServerOpt 

type ServerOpt func(*Server)

ServerOpt configures Server

func WithCachePolicy 

func WithCachePolicy(disableCache bool) ServerOpt

WithCachePolicy will pass cache option to server

func WithConstLabel 

func WithConstLabel(labels prometheus.Labels) ServerOpt

WithConstLabel copy constant label map to server

func WithQueries 

func WithQueries(queries map[string]*Query) ServerOpt

WithQueries set server's default query set

func WithServerConnectTimeout 

func WithServerConnectTimeout(timeout int) ServerOpt

WithServerConnectTimeout will set a connect timeout for server precheck queries otherwise, a default value 100ms will be used. Increase this value if you are monitoring a remote (cross-DC, cross-AZ) instance

func WithServerTags 

func WithServerTags(tags []string) ServerOpt

WithServerTags will mark server only execute query without cluster tag

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.