Salt Module: NodeCFG

This module worked similar to pillar.get() except that it was designed to operate on a shared pillar dictionary. It was rewritten because the old version had many bugs and broke the standard assumptions. This temporary module was eventually deprecated in favor of a system that doesn’t use shared dictionaries.

Original Version

This is not the original version. There were too many bugs that needed urgent fixing and then a repository reset lost the original.

# -*- coding: utf-8 -*-
'''
Custom module to lookup node config in pillar, if not found,
fall back to defaults values, if not found KeyError
'''
from __future__ import absolute_import

import logging

# Import salt libs
import salt
import salt.utils
from salt.defaults import DEFAULT_TARGET_DELIM
log = logging.getLogger(__name__)

def get(key, default=KeyError, default_lookup=True, missing_ok=False):
    delimiter = DEFAULT_TARGET_DELIM
    node_id = __grains__['id']
    node_key = ':'.join(("nodes", node_id, key))
    defaults_key = DEFAULT_TARGET_DELIM.join(("nodes{}defaults".format(DEFAULT_TARGET_DELIM), key))

    log.debug("Node CFG Module: {0}/{1}".format(node_key, defaults_key))

    ret = salt.utils.traverse_dict_and_list(__pillar__, node_key, KeyError, delimiter)
    log.debug("Node Lookup key ({0}): {1}".format(node_key, ret))

    if default_lookup is True:
        if ret is KeyError or ret is None:
            ret = salt.utils.traverse_dict_and_list(__pillar__, defaults_key, KeyError, delimiter)
            log.debug("Default Lookup key ({0}): {1}".format(defaults_key, ret))
    else:
        log.debug("skipping default lookup.")

    # assign the default value because we found nothing
    if ret is KeyError and default is not KeyError:
        ret = default

    # if we don't pass a default value, we stop because it's a good guess that the lookup key is wrong
    # and we'll want to know
    if ret is KeyError and not missing_ok:
        raise KeyError("Node pillar keys not found: {0} or {1}".format(node_key, defaults_key))

    return ret

Biggest Issues

.get() behavior is extremely non-standard (was .get(hostname, key, default, missing_ok))
Confusing .get() usage created hidden bugs across entire repository
- Often swapped default value and lookup hostname
- Lots of changing default in every config
Required every single pillar value (secret) to be exposed to every single host
Lots of code that didn’t do what was described
Imported delimeter but never used it

New Version

#!/usr/bin/env python
'''
Custom module to look up node config in pillar, if not found,
fall back to defaults values, if not found then return None.
'''


##
# TODO:
# - This should return None when nothing is found
# - missing_ok should be removed; replace with default=None
# ^^ requires first updating states
##
def get(key, default=KeyError, default_lookup=True, missing_ok=False):
    result = __salt__['pillar.get']('nodes:{}:{}'.format(__grains__['id'], key), 'not-found')
    if result == 'not-found' and default_lookup:
        result = __salt__['pillar.get']('nodes:defaults:{}'.format(key), default)
    elif result == 'not-found':
        result = default

    if result is KeyError and not missing_ok:
        # legacy logic: assume at this point that the lookup key is probably wrong
        raise KeyError("Node pillar keys not found: {0}".format(key))

    return result