Specs¶

spec_base is the core of input_algorithms and implements many specification classes based off the Spec class.

A specification is an object with a normalise method and is used to validate and transform data.

NotSpecified¶

class input_algorithms.spec_base.NotSpecified¶: Tell the difference between None and not specified

Specifications¶

class input_algorithms.spec_base.Spec(*pargs, **kwargs)¶

Default shape for a spec (specification, not test!)

__init__

passes all *args and **kwargs to a self.setup if that exists on the class

normalise

Calls one of the following functions on the class, choosing in this order:

normalise_either

If defined, then this is meant to work with both a specified value as well as NotSpecified.

If this function returns NotSpecified, then we continue looking for a function to handle this value.

normalise_empty

Called if the value is NotSpecified.

default

Called if the value is NotSpecified.

return NotSpecified

If the value is NotSpecified and the above don’t return, then the value is returned as is.

normalise_filled

The value is not NotSpecified

If none of those options are defined, then an error is raised complaining that we couldn’t work out what to do with the value.

Note that any validation errors should be an subclass of input_algorithms.errors.BadSpec. This is because the default specs only catch such exceptions. Anything else is assumed to be a ProgrammerError and worthy of a traceback.

fake_filled

If fake is on the class that is used, otherwise if default is on the class that is used, otherwise we return NotSpecified.

This is used to generate fake data from a specification.

The idea is that a Spec is an object with a normalise method that takes in two objects: meta and val.

meta

Should be an instance of input_algorithms.meta.Meta and is used to indicate where we are. This should be passed to any children specifications.

For example, if we are normalising a dictionary where a child specification is used on a particular value at a key called a_key, we should do something similar to:

child_spec().normalise(meta.at("a_key"), val["a_key"])

val

Should be the value we are normalising. The normalising process should validate and transform the value to be what we are specifying.

For example listof(string_spec()) will transform a single string to be a list of one string.

When you create a subclass of Spec you either implement one of the normalise_* methods or normalise itself.

apply_validators¶

input_algorithms.spec_base.apply_validators(meta, val, validators, chain_value=True)¶

Apply a number of validators to a value.

chain_value: Sets whether to pass the validated value into the next validator or whether to use the original value each time

All validators are tried and errors are collected.

If any fails, an error is raised, otherwise the value is returned.

Where value is the original value, or the result of the last validator depending on chain_value.

Available Specs¶

pass_through_spec¶

Usage
pass_through_spec().normalise(meta, val)
Will not touch the value in any way and just return it.

always_same_spec¶

Usage
always_same_spec(result).normalise(meta, val)
Will ignore value and just return result.

dictionary_spec¶

Usage
dictionary_spec().normalise(meta, val)
Will normalise NotSpecified into {}

Specified values are valid if type(val) is dict or val.is_dict() is True. If either those conditions are true, then the dictionary is returned as is.

dictof¶

Usage
dictof(name_spec, value_spec).normalise(meta, val)

# or

dictof(name_spec, value_spec, nested=True).normalise(meta, val)
This will first use dictionary_spec logic on the value to ensure we are normalising a dictionary.

It will then use name_spec and value_spec on the items in the value to produce a resulting dictionary.

For example if we have {"a": 1, "b": 2}, using dictof is equivalent to:
{ name_spec.normalise(meta.at("a"), "a"): value_spec.normalise(meta.at("a"), 1)
, name_spec.normalise(meta.at("b"), "b"): value_spec.normalise(meta.at("b"), 2)
}
This specification will also do the same to any dictionary values it has if nested is set to True (defaults to False)

It will also collect any errors and raise a collection of all errors it comes across.

tupleof¶

Usage
tupleof(spec).normalise(meta, val)
This specification will transform NotSpecified into ()

If we don’t have a tuple, we turn the value into a tuple of that value. Except for lists which are turned in a tuple.

The resulting tuple of items is returned.

listof¶

Usage
listof(spec).normalise(meta, val)

# or

listof(spec, expect=typ).normalise(meta, val)
This specification will transform NotSpecified into []

If we don’t have a list, we turn the value into a list of that value.

If expect is specified, any item already passing isinstance(item, expect) is left alone, otherwise spec is used to normalise the value.

The resulting list of items is returned.

set_options¶

Usage
set_options(<key1>=<spec1>, ..., <keyn>=<specn>).normalise(meta, val)

# For example

set_options(key_1=integer_spec(), key_2=string_spec()).normalise(meta, val)
This specification transforms NotSpecified into {}.

Specified values are validated using dictionary_spec and then for each keyword argument we use either that key in the val or Notspecified and normalise it with the spec for that key.

Finally, we assemble the result from all these keys in a dictionary and return that.

Errors are collected and raised in a group.

Extra keys in val are ignored.

defaulted¶

Usage
defaulted(spec, dflt).normalise(meta, val)
This specification will return dflt if val is NotSpecified.

Otherwise, it merely proxies spec and does spec.normalise(meta, val)

required¶

Usage
required(spec).normalise(meta, val)
This specification will raise an error if val is NotSpecified.

Otherwise, it merely proxies spec and does spec.normalise(meta, val)

boolean¶

Usage
boolean().normalise(meta, val)
This complains if the value is not isintance(val, bool).

Otherwise it just returns the val.

Note

This specification does not handle NotSpecified. This is a deliberate decision. Use defaulted(boolean(), <dflt>) if you want to handle that.

directory_spec¶

Usage
directory_spec().normalise(meta, val)

# or

directory_spec(spec).normalise(meta, val)
This specification will first normalise val with spec if spec is specified.

It then makes sure that val is a string, exists, and is a directory.

If it isn’t, an error is raised, otherwise the val is returned.

filename_spec¶

Usage
directory_spec().normalise(meta, val)

# or

filename_spec(spec).normalise(meta, val)
This specification will first normalise val with spec if spec is specified.

It then makes sure that val is a string, exists, and is a file.

If it isn’t, an error is raised, otherwise the val is returned.

file_spec¶

Usage
file_spec().normalise(meta, val)
This will complain if val is not a file object, otherwise it just returns val.

string_spec¶

Usage
string_spec().normalise(meta, val)
This transforms NotSpecified into ""

If val is specified, it will complain if not isinstance(val, six.string_types) , otherwise it just returns val.

integer_spec¶

Usage
integer_spec().normalise(meta, val)
This will complain if val is not an integer, unless it has isdigit and this function returns True.

We return int(val) regardless.

Note

This specification does not handle NotSpecified. This is a deliberate decision. Use defaulted(integer_spec(), <dflt>) if you want to handle that.

float_spec¶

Usage
float_spec().normalise(meta, val)
If the val is not a bool then we do float(val) and return the result.

Otherwise, or if that fails, an error is raised.

string_or_int_as_string_spec¶

Usage
string_or_int_as_string_spec().normalise(meta, val)
This transforms NotSpecified into ""

If the val is not an integer or string, it will complain, otherwise it returns str(val).

valid_string_spec¶

Usage
valid_string_spec(validator1, ..., validatorn).normalise(meta, val)
This takes in a number of validator specifications and applies them to val after passing through valid_string_spec logic.

Validators are just objects with normalise methods that happen to raise errors and return the val as is.

If none of the validators raise an error, the original val is returned.

integer_choice_spec¶

Usage
integer_choice_spec([1, 2, 3]).normalise(meta, val)

# or

integer_choice_spec([1, 2, 3], reason="Choose one of the first three numbers!").normalise(meta, val)
This absurdly specific specification will make sure val is an integer before making sure it’s one of the choices that are provided.

It defaults to complaining Expected one of the available choices unless you provide reason, which it will use instead if it doesn’t match one of the choices.

string_choice_spec¶

Usage
string_choice_spec(["a", "b", "c"]).normalise(meta, val)

# or

string_choice_spec(["a", "b", "c"], reason="Choose one of the first three characters in the alphabet!").normalise(meta, val)
This absurdly specific specification will make sure val is a string before making sure it’s one of the choices that are provided.

It defaults to complaining Expected one of the available choices unless you provide reason, which it will use instead if it doesn’t match one of the choices.

create_spec¶

Usage
create_spec(
    kls
    , validator1, ..., validatorn
    , key1=spec1, ..., keyn=specn
    ).normalise(meta, val)
This specification will return val as is if it’s already an instance of kls.

Otherwise, it will run val through all the validator``s before using the ``key->``spec`` keyword arguments in a set_options specification to create the arguments used to instantiate an instance of kls.

or_spec¶

Usage
or_spec(spec1, ..., specn).normalise(meta, val)
This will keep trying spec.normalise(meta, val) until it finds one that doesn’t raise a BadSpec error.

If it can’t find one, then it raises all the errors as a group.

match_spec¶

Usage
match_spec((typ1, spec1), ..., (typn, specn)).normalise(meta, val)

# or

match_spec((typ1, spec1), ..., (typn, specn), fallback=fspec).normalise(meta, val)
This will find the spec associated with the first typ that succeeds isinstance(val, typ).

Note

If spec is callable, we do spec().normalise(meta, val).

If fallback is specified and none of the typ``s match the ``val then fpsec is used as the spec. It is also called first if it’s a callable object.

If we can’t find a match for the val, an error is raised.

and_spec¶

Usage
and_spec(spec1, ..., specn).normalise(meta, val)
This will do val = spec.normalise(meta, val) for each spec that is provided and returns the final val.

If any of the ``spec``s fail, then an error is raised.

optional_spec¶

Usage
optional_spec(spec).normalise(meta, val)
This will return NotSpecified if the val is NotSpecified.

Otherwise it merely acts as a proxy for spec.

dict_from_bool_spec¶

Usage
dict_from_bool_spec(dict_maker, spec).normalise(meta, val)
If val is NotSpecified then we do spec.normalise(meta, {})

If val is a boolean, we first do val = dict_maker(meta, val) and then do spec.normalise(meta, val) to return the value.

Example:
A good example is setting enabled on a dictionary:
spec = dict_from_bool_spec(lambda b: {"enabled": b}, set_options(enabled=boolean()))

spec.normalise(meta, False) == {"enabled": False}

spec.normalise(meta, {"enabled": True}) == {"enabled": True}

formatted¶

Usage
formatted(spec, formatter).normalise(meta, val)

# or

formatted(spec, formatter, expected_type=typ).normalise(meta, val)

# or

formatted(spec, formatter, after_format=some_spec()).normlise(meta, val)
This specification is a bit special and is designed to be used with MergedOptionStringFormatter from the option_merge library (http://option-merge.readthedocs.org/en/latest/docs/api/formatter.html).

The idea is that meta.everything is an instance of MergedOptions and it will create a new instance of meta.everything.__class__ using meta.everything.converters and meta.everything.dont_prefix if they exist. Note that this should work with normal dictionaries as well.

We then update our copy of meta.everything with meta.key_names() and create an instance of formatter using this copy of the meta.everything , meta.path and spec.normalise(meta, val) as the value.

We call format on the formatter instance, check that it’s an instance of expected_type if that has been specified.

Once we have our formatted value, we normalise it with after_format if that was specified.

And finally, return a value!

many_format¶

Usage
many_format(spec, formatter).normalise(meta, val)

# or

many_format(spec, formatter, expected_type=typ).normalise(meta, val)
This is a fun specification!

It essentially does formatted(spec, formatter, expected_type=typ).normalise(meta, val) until the result doesn’t change anymore.

Before doing the same thing on "{{{val}}}".format(val)

Example:
Let’s say we’re at images.my_image.persistence.image_name in the configuration.

This means {_key_name_2} (which is from meta.key_names()) is equal to my_image.
many_format(overridden("images.{_key_name_2}.image_name"), formatter=MergedOptionStringFormatter)

# Is equivalent to

formatted(overridden("{images.my_image.image_name}"), formatter=MergedOptionStringFormatter)
This essentially means we can format a key in the options using other keys from the options!

overridden¶

Usage
overridden(value).normalise(meta, val)
This will return value regardless of what val is!

any_spec¶

Usage
any_spec().normalise(meta, val)
Will return val regardless of what val is.

container_spec¶

Usage
container_spec(kls, spec).normalise(meta, val)
This will apply spec.normalise(meta, val) and call kls with the result of that as the one argument.

Note

if the val is already isinstance(val, kls) then it will just return val.

delayed¶

Usage
delayed(spec).normalise(meta, val)
This returns a function that when called will do spec.normalise(meta, val)

typed¶

Usage
typed(kls).normalise(meta, val)
This will return the value as is as long as it’s isinstance of kls

Otherwise it complains that it’s the wrong type

has¶

Usage
has(prop1, prop2, ..., propn).normalise(meta, val)
This will complain if the value does not have any of the specified properties (using hasattr)

tuple_spec¶

Usage
tuple_spec(spec1, spec2, ..., specn).normalise(meta, val)
Will complain if the value is not a tuple or doesn’t have the same number of items as specified specs.

Will complain if any of the specs fail for their respective part of val.

Returns the result of running all the values through the specs as a tuple.

none_spec¶

Usage
none_spec().normalise(meta, val)
Will complain if the value is not None. Otherwise returns None.

Defaults to None.

many_item_formatted_spec¶

Usage

class FinalKls(dictobj):
    fields = ["one", "two", ("three", None)]

class my_special_spec(many_item_formatted_spec):
    specs = [integer_spec(), string_spec()]

    def create_result(self, one, two, three, meta, val, dividers):
        if three is NotSpecified:
            return FinalKls(one, two)
        else:
            return FinalKls(one, two, three)

    # The rest of the options are optional
    creates = FinalKls
    value_name = "special"
    seperators = "^"
    optional_specs = [boolean()]

spec = my_special_spec()
spec.normalise(meta, "1^tree") == FinalKls(1, "tree")
spec.normalise(meta, [1, "tree"]) == FinalKls(1, "tree")
spec.normalise(meta, [1, tree, False]) == FinalKls(1, "tree", False)

We can also define modification hooks for each part of the spec:

class my_special_spec(many_item_formatted_spec):
    specs = [integer_spec(), integer_spec(), integer_spec()]

    def spec_wrapper_2(self, spec, one, two, meta, val, dividers):
        return defaulted(spec, one + two)

    def determine_2(self, meta, val):
        return 42

    def alter_2(self, one, meta, original_two, val):
        if one < 10:
            return original_two
        else:
            return original_two * 10

    def alter_3(self, one, two, meta, original_three, val):
        if two < 100:
            return original_three
        else:
            return original_three * 100

    def create_result(self, one, two, three, meta, val, dividers):
        return FinalKls(one, two, three)

A spec for something that is many items Either a list or a string split by “:”

If it’s a string it will split by ‘:’ Otherwise if it’s a list, then it will use as is and will complain if it has two many values

It will use determine_<num> on any value that is still NotSpecified after splitting the val.

And will use alter_<num> on all values after they have been formatted.

Where <num> is 1 indexed index of the value in the spec specifications.

Finally, create_result is called at the end to create the final result from the determined/formatted/altered values.