README
¶
DS
Package ds (short for "data store") is a key-value store with hash indexes. It allows for rudimentary but lightning fast retrieval of grouped or relevant data without having to iterate over all objects in the store.
Define the primary key, indexed keys, and unique keys as tags on struct fields, and DS takes care of the rest.
Usage & Examples
Examples can be found on the documentation for the library
Documentation
¶
Overview ¶
Package ds (short for "data store") is a key-value store with hash indexes. It allows for rudimentary but lightning fast retrieval of grouped or relevant data without having to iterate over all objects in the store.
Define the primary key, indexed keys, and unique keys as tags on struct fields, and DS takes care of the rest.
Index ¶
Examples ¶
Constants ¶
const ( ErrMissingRequiredValue = "missing required value" ErrPointer = "pointer provided when none expected" ErrPropertyChanged = "property changed" ErrDuplicatePrimaryKey = "duplicate value for primary key" ErrDuplicateUnique = "duplicate value for unique field" ErrFieldNotIndexed = "field not indexed" ErrFieldNotUnique = "field not unique" ErrMigrateTablePathNotFound = "table path" ErrBadTableFile = "bad table file" )
Error constant
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
Fields []Field
TypeOf string
PrimaryKey string
Indexes []string
Uniques []string
LastInsertIndex uint64
Version int
}
Config describes ds table configuration
type GetOptions ¶
type GetOptions struct {
// Should the results be sorted. Does nothing for unsorted tables
Sorted bool
// If results are to be sorted, should they be from most recent to oldest (true) or invese (false)
Ascending bool
// The maximum number of entries to return. 0 means unlimited
Max int
}
GetOptions describes options for getting entries from a DS table
type IReadTransaction ¶ added in v1.8.0
type IReadTransaction interface {
// Get will get a single entry by its primary key. Returns (nil, nil) if nothing found.
Get(primaryKey interface{}) (interface{}, error)
// GetIndex will get multiple entries that contain the same value for the specified indexed field.
// Result is not ordered. Use GetIndexSorted to return a sorted slice.
// Returns an empty array if nothing found.
GetIndex(fieldName string, value interface{}, options *GetOptions) ([]interface{}, error)
// GetUnique will get a single entry based on the value of the provided unique field.
// Returns (nil, nil) if nothing found.
GetUnique(fieldName string, value interface{}) (interface{}, error)
// GetAll will get all of the entries in the table.
GetAll(options *GetOptions) ([]interface{}, error)
}
IReadTransaction describes an interface for performing read-only operations on a table.
type IReadWriteTransaction ¶ added in v1.8.0
type IReadWriteTransaction interface {
IReadTransaction
// Add will add a new object to the table. o must the the same type that was used to register the table and cannot be a pointer.
Add(o interface{}) error
// Delete will delete the provided object and clean indexes
Delete(o interface{}) error
// DeletePrimaryKey will delete the object with the associated primary key and clean indexes. Does nothing if not object
// matches the given primary key.
DeletePrimaryKey(o interface{}) error
// DeleteUnique will delete the object with the associated unique value and clean indexes. Does nothing if no object
// matched the given unique fields value.
DeleteUnique(field string, o interface{}) error
// DeleteAllIndex will delete all objects matching the given indexed fields value
DeleteAllIndex(fieldName string, value interface{}) error
// DeleteAll delete all objects from the table
DeleteAll() error
// Update will update an existing object in the table. The primary key must match for this object
// otherwise it will just be inserted as a new object. Updated objects do not change positions in a sorted
// table.
Update(o interface{}) error
}
IReadWriteTransaction describes an interface for performing read or write operations on a table.
type MigrateParams ¶
type MigrateParams struct {
// TablePath the path to the existing table file
TablePath string
// NewPath the path for the new table file. This can be the same as the old table.
NewPath string
// OldType an instance of a struct object that has the same definition as the existing table.
OldType interface{}
// NewType an instance of a struct object that has the definition that shall be used. This can be the same as the
// OldType.
NewType interface{}
// DisableSorting (optional) if the current table is sorted, set this to true to disable sorting
// Note: This is irreversible!
DisableSorting bool
// MigrateObject method called for each entry in the table in reverse order. Return a new type, error, or nil.
// Migration is halted if an error is returned.
// Return (nil, nil) and the entry will be skipped from migration, but migration will continue.
MigrateObject func(o interface{}) (interface{}, error)
// KeepBackup (optional) if false the backup copy of the table will be discarded if the migration was successful. If true
// the copy is not deleted.
KeepBackup bool
}
MigrateParams describes the parameters to perform a DS table migration. All fields are required unless otherwise specified.
type MigrationResults ¶
type MigrationResults struct {
// Success was the migration successful
Success bool
// Error if unsuccessful, this will be the error that caused the failure
Error error
// EntriesMigrated the number of entries migrated
EntriesMigrated uint
// EntriesSkipped the number of entries skipped
EntriesSkipped uint
}
MigrationResults describes results from a migration
func Migrate ¶
func Migrate(params MigrateParams) (results MigrationResults)
Migrate will migrate a DS table from one object type to another. You must migrate if the old data type is not compatible with the new type, such as if an existing field was changed. You don't need to migrate if you add or remove an existing field.
Before the existing data is touched, a copy is made with "_backup" appended to the filename, and a new table file is created with the migrated entries. Upon successful migration, the backup copy is deleted (by default). If the table being migrated is sorted, the original order is preserved.
Ensure you read the documentation of the MigrateParams struct, as it goes into greater detail on the parameters required for migration, and what they do.
Example ¶
package main
import (
"fmt"
"github.com/ecnepsnai/ds"
)
func main() {
// Define a struct that maps to the current type used in the table
type oldType struct {
Username string `ds:"primary"`
Email string `ds:"unique"`
FirstName string
LastName string
}
// Define your new struct
type newType struct {
Username string `ds:"primary"`
Email string `ds:"unique"`
Name string
}
// In this example, we're merging the "FirstName" and "LastName" fields of the User object to
// just a single "Name" field
result := ds.Migrate(ds.MigrateParams{
TablePath: "/path/to/table.db",
NewPath: "/path/to/table.db", // You can specify the same path, or a new one if you want
OldType: oldType{},
NewType: newType{}, // NewType can be the same as the old type if you aren't changing the struct
MigrateObject: func(o interface{}) (interface{}, error) {
old := o.(oldType)
// Within the MigrateObject function you can:
// 1. Return a object of the NewType (specified in the MigrateParams)
// 2. Return an error and the migration will abort
// 3. Return nil and this entry will be skipped
return newType{
Username: old.Username,
Email: old.Email,
Name: old.FirstName + " " + old.LastName,
}, nil
},
})
if !result.Success {
// Migration failed.
panic(result.Error)
}
fmt.Printf("Migration successful. Entries migrated: %d, skipped: %d\n", result.EntriesMigrated, result.EntriesSkipped)
}
Output:
type Options ¶
type Options struct {
// DisableSorting disable all sorting features. This will make tables smaller, and inserts/removes/deletes faster.
DisableSorting bool
// contains filtered or unexported fields
}
Options describes options for DS tables. Once set, these cannot be changed.
type Table ¶
type Table struct {
Name string
// contains filtered or unexported fields
}
Table describes a ds table. A table is mapped to a single registered object type and contains both the data and the indexes.
func Register ¶
Register will register an instance of a struct with ds, creating a table (or opening an existing table) for this type at the specified file path.
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
// Primary fields represent the primary key of the object. Your object must have exactly one primary field
// and its value is unique
Username string `ds:"primary"`
// Unique fields function just like primary fields except any field (other than the primary field) can be unique
Email string `ds:"unique"`
// Index fields represent fields where objects with identical values are grouped together so they can be fetched
// quickly later
Enabled bool `ds:"index"`
// Fields with no ds tag are saved, but you can't fetch based on their value, and can have duplicate values
// between entries
Password string
}
tablePath := "user.db"
table, err := ds.Register(User{}, tablePath, nil)
if err != nil {
panic(err)
}
// Don't forget to close your table when you're finished
table.Close()
}
Output:
func (*Table) Close ¶
func (table *Table) Close()
Close will close the table. This will not panic if the table has not been opened or already been closed.
func (*Table) IsIndexed ¶
IsIndexed is the given field indexed
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
Username string `ds:"primary"`
Email string `ds:"email"`
Enabled bool `ds:"index"`
}
tablePath := "user.db"
table, err := ds.Register(User{}, tablePath, nil)
if err != nil {
panic(err)
}
table.IsIndexed("Username") // returns False
table.IsIndexed("Enabled") // returns True
}
Output:
func (*Table) IsUnique ¶
IsUnique is the given field unique
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
Username string `ds:"primary"`
Email string `ds:"email"`
Enabled bool `ds:"index"`
}
tablePath := "user.db"
table, err := ds.Register(User{}, tablePath, nil)
if err != nil {
panic(err)
}
table.IsUnique("Username") // returns False
table.IsUnique("Email") // returns True
}
Output:
func (*Table) StartRead ¶ added in v1.8.0
func (table *Table) StartRead(ctx func(tx IReadTransaction) error) error
StartRead will start a new read-only transaction on the table. ctx will be called when the transaction is ready. This method may block if there is an active write transaction.
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
Username string `ds:"primary"`
Password string
Email string `ds:"unique"`
Enabled bool `ds:"index"`
}
var table *ds.Table // Assumes the table is already registered, see ds.Register for an example
// Get the user with username "example"
var user *User
table.StartRead(func(tx ds.IReadTransaction) error {
object, err := tx.Get("example")
if err != nil {
return err // error fetching data
}
if object == nil {
return nil // no object found
}
u, ok := object.(User)
if !ok {
panic("incorrect type") // data in table was not the same type as expected
}
user = &u
return nil
})
if user == nil {
panic("No user found!")
}
// Use the user object
}
Output:
func (*Table) StartWrite ¶ added in v1.8.0
func (table *Table) StartWrite(ctx func(tx IReadWriteTransaction) error) error
StartWrite will start a new read-write transaction on the table. ctx will be called when the transaction is ready. This method may block if there is an active transaction.
Note that any any errors or partial changes made during ctx are not reverted.
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
Username string `ds:"primary"`
Password string
Email string `ds:"unique"`
Enabled bool `ds:"index"`
}
var table *ds.Table // Assumes the table is already registered, see ds.Register for an example
newUser := User{
Username: "ian",
Password: "hunter2",
Email: "email@domain",
Enabled: true,
}
err := table.StartWrite(func(tx ds.IReadWriteTransaction) error {
return tx.Add(newUser)
})
if err != nil {
panic(err)
}
}
Output:
Source Files
Directories