collector

package
v0.0.0-...-6e6aaf7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 14, 2025 License: Apache-2.0 Imports: 29 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AggregateMetricsCollector 

type AggregateMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
}

AggregateMetricsCollector is a prometheus.Collector that submits dynamically-calculated aggregate metrics about scraping progress.

func (*AggregateMetricsCollector) Collect 

func (c *AggregateMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*AggregateMetricsCollector) Describe 

func (c *AggregateMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

type CapacityPluginMetricsCollector 

type CapacityPluginMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// When .Override is set, the DB is bypassed and only the given
	// CapacityPluginMetricsInstances are considered. This is used for testing only.
	Override []CapacityPluginMetricsInstance
}

CapacityPluginMetricsCollector is a prometheus.Collector that submits metrics which are specific to the selected capacity plugins.

func (*CapacityPluginMetricsCollector) Collect 

func (c *CapacityPluginMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*CapacityPluginMetricsCollector) Describe 

func (c *CapacityPluginMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

type CapacityPluginMetricsInstance 

type CapacityPluginMetricsInstance struct {
	CapacitorID       string
	SerializedMetrics string
}

CapacityPluginMetricsInstance describes a single project service for which plugin metrics are submitted. It appears in type CapacityPluginMetricsCollector.

type Collector 

type Collector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// Usually logg.Error, but can be changed inside unit tests.
	LogError func(msg string, args ...any)
	// Usually time.Now, but can be changed inside unit tests.
	// MeasureTimeAtEnd behaves slightly differently in unit tests: It will advance
	// the mock.Clock before reading it to simulate time passing during the previous task.
	MeasureTime      func() time.Time
	MeasureTimeAtEnd func() time.Time
	// Usually addJitter, but can be changed inside unit tests.
	AddJitter func(time.Duration) time.Duration
}

Collector provides methods that implement the collection jobs performed by limes-collect. The struct contains references to the driver used, the plugin (which defines the service type to be targeted), and a few other things; basically everything that needs to be replaced by a mock implementation for the collector's unit tests.

func NewCollector 

func NewCollector(cluster *core.Cluster, dbm *gorp.DbMap) *Collector

NewCollector creates a Collector instance.

func (*Collector) ApplyQuotaOverridesJob 

func (c *Collector) ApplyQuotaOverridesJob(registerer prometheus.Registerer) jobloop.Job

ApplyQuotaOverridesJob is a jobloop.CronJob.

It loads quota overrides from the respective config file and updates the `project_resources.override_quota_from_config` column to match the configured values.

func (*Collector) CapacityScrapeJob 

func (c *Collector) CapacityScrapeJob(registerer prometheus.Registerer) jobloop.Job

CapacityScrapeJob is a jobloop.Job. Each task scrapes one capacitor. Cluster resources managed by this capacitor are added, updated and deleted as necessary.

func (*Collector) CheckConsistencyJob 

func (c *Collector) CheckConsistencyJob(registerer prometheus.Registerer) jobloop.Job

func (*Collector) CleanupOldCommitmentsJob 

func (c *Collector) CleanupOldCommitmentsJob(registerer prometheus.Registerer) jobloop.Job

CleanupOldCommitmentsJob is a jobloop.CronJob.

It moves expired commitments to state "expired" and cleans up old expired commitments that do not have any non-expired predecessors.

func (*Collector) ExpiringCommitmentNotificationJob 

func (c *Collector) ExpiringCommitmentNotificationJob(registerer prometheus.Registerer) jobloop.Job

ExpiringCommitmentNotificationJob is a jobloop.Job. A task scrapes commitments that are or are about to expire. For all applicable commitments within a project the mail content to inform customers will be prepared and added to a queue. Long-term commitments will be queued while short-term commitments will only be marked as notified.

func (*Collector) MailDeliveryJob 

func (c *Collector) MailDeliveryJob(registerer prometheus.Registerer, client MailClient) jobloop.Job

MailDeliveryJob is a jobloop.CronJob. A task searches for a queued mail notification. If any is found, it builds the mail request body and posts the mail to the mail API. Unsuccessful mail deliveries will increase the fail counter and will be requeued with an updated submission timestamp.

func (*Collector) RateScrapeJob 

func (c *Collector) RateScrapeJob(registerer prometheus.Registerer) jobloop.Job

RateScrapeJob looks at one specific project service per task, checks the database for outdated or missing rate records for the given service, and updates them by querying the backend service.

This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.

func (*Collector) ResourceScrapeJob 

func (c *Collector) ResourceScrapeJob(registerer prometheus.Registerer) jobloop.Job

ResourceScrapeJob looks at one specific project service per task, collects quota and usage information from the backend service, and adjusts the backend quota if it differs from the desired values.

This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.

func (*Collector) ScanDomains 

func (c *Collector) ScanDomains(ctx context.Context, opts ScanDomainsOpts) (result []string, resultErr error)

ScanDomains queries Keystone to discover new domains, and returns a list of UUIDs for the newly discovered domains.

func (*Collector) ScanDomainsAndProjectsJob 

func (c *Collector) ScanDomainsAndProjectsJob(registerer prometheus.Registerer) jobloop.Job

ScanDomainsAndProjectsJob is a jobloop.CronJob. It syncs domains and projects from Keystone into the Limes database.

func (*Collector) ScanProjects 

func (c *Collector) ScanProjects(ctx context.Context, domain *db.Domain) (result []string, resultErr error)

ScanProjects queries Keystone to discover new projects in the given domain.

func (*Collector) SyncQuotaToBackendJob 

func (c *Collector) SyncQuotaToBackendJob(registerer prometheus.Registerer) jobloop.Job

SyncQuotaToBackendJob looks for project services that need to have their quota applied to the backend, and runs SetQuota for those services.

This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.

type DataMetricsReporter 

type DataMetricsReporter struct {
	Cluster      *core.Cluster
	DB           *gorp.DbMap
	ReportZeroes bool
}

DataMetricsReporter renders Prometheus metrics for data attributes (quota, usage, etc.) for all projects known to Limes.

It is an http.Handler, instead of implementing the prometheus.Collector interface (like all the other Collector types in this package) and going through the normal promhttp facility.

We are not going through promhttp here because promhttp insists on holding all metrics in memory before rendering them out (in order to sort them). Given the extremely high cardinality of these metrics, this results in unreasonably high memory usage spikes.

This implementation also holds all the metrics in memory (because ORDER BY on database level turned out to be prohibitively expensive), but we hold their rendered forms (i.e. something like `{bar="bar",foo="foo"} 42` instead of a dozen allocations for each label name, label value, label pair, a map of label pairs, and so on) in order to save memory.

func (*DataMetricsReporter) ServeHTTP 

func (d *DataMetricsReporter) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements the http.Handler interface.

type MailClient 

type MailClient interface {
	// Builds the request to sent the mail content to a mail API.
	PostMail(ctx context.Context, req MailRequest) error
}

MailClient is an interface that provides the methods to communicate with a mail backend service.

func NewMailClient 

func NewMailClient(provider *gophercloud.ProviderClient, endpoint string) (MailClient, error)

NewMailClient returns a service client to communicate with a mail API.

type MailRequest 

type MailRequest struct {
	ProjectID string `json:"project_id"`
	Subject   string `json:"subject"`
	MimeType  string `json:"mime_type"`
	MailText  string `json:"mail_text"`
}

type QuotaPluginMetricsCollector 

type QuotaPluginMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// When .Override is set, the DB is bypassed and only the given
	// QuotaPluginMetricsInstances are considered. This is used for testing only.
	Override []QuotaPluginMetricsInstance
}

QuotaPluginMetricsCollector is a prometheus.Collector that submits metrics which are specific to the selected quota plugins.

func (*QuotaPluginMetricsCollector) Collect 

func (c *QuotaPluginMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*QuotaPluginMetricsCollector) Describe 

func (c *QuotaPluginMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

type QuotaPluginMetricsInstance 

type QuotaPluginMetricsInstance struct {
	Project           core.KeystoneProject
	ServiceType       db.ServiceType
	SerializedMetrics string
}

QuotaPluginMetricsInstance describes a single project service for which plugin metrics are submitted. It appears in type QuotaPluginMetricsCollector.

type ScanDomainsOpts 

type ScanDomainsOpts struct {
	// Recurse into ScanProjects for all domains in the selected cluster,
	// rather than just for new domains.
	ScanAllProjects bool
}

ScanDomainsOpts contains additional options for ScanDomains().

type TaskTiming 

type TaskTiming struct {
	StartedAt  time.Time // filled during DiscoverTask
	FinishedAt time.Time // filled during ProcessTask
}

TaskTiming appears in the task types of our ProducerConsumerJobs.

func (TaskTiming) Duration 

func (t TaskTiming) Duration() time.Duration

Duration measures the duration of the main portion of a task.

type UndeliverableMailError 

type UndeliverableMailError struct {
	Inner error
}

UndeliverableMailError is a custom error type to define udeliverable mails. Used in the MailClient interface implementations.

func (UndeliverableMailError) Error 

func (e UndeliverableMailError) Error() string

implements https://pkg.go.dev/builtin#error

func (UndeliverableMailError) Unwrap 

func (e UndeliverableMailError) Unwrap() error

implements the interface implied by https://pkg.go.dev/errors

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.