Holy crap it's live: JSONConfigParser 0.1.0

It only took about two hours of fiddle faddling and there's probably more commits from trying to get it to do the thing than anything else, but I finally pushed my first package to PyPi. :) I still need to add

jsonconfigparser 0.1.0 [pypi] [github]

Technically it's 0.1.1 but that's only because I didn't know about MANIFEST.in Pull requests, patches and feature requests are all more than welcome!
I suppose I should take the time to describe it a little. The tag line is "Quick and easy editting of JSON files." which is probably a bit of a misnomer but it beats editting files by hand, in my opinion. There's three ways of interacting with it:

• CLI through your shell
• An interactive prompt
• Through Python itself

CLI Implementation

Getting here is as simple as typing jsonconf. It expects two positional arguments:

• The first is a path to a json file.
• The second is the action you want to take on the file. They're detailed on the github and pypi pages but I'll go over the actions and flags a little.

Actions

These are the main actions used by the command line interface

• addfile:
  • Uses the -o/--other flag to update the specified json file with another. Using this command will overwrite existing keys with the values in the appended file if there are shared keys.
  • Ex: jsonconf path/to/conf.json addfile -o path/to/other.json
• addfield:
  • Adds a field to the specified json file at a specified end point, optionally converts the value to a non-string type
  • Ex: jsonconf path/to/conf.json addfield -p $.age -v 25 -c int
• append:
  • Adds a value to a JSON collection at the specified endpoint, optionally converts the value to a non-string type, optionally affects every found path found.
  • Ex: jsonconf path/to/conf.json append -p $.packages -v jsonconfigparser
  • Ex: jsonconf path/to/conf.json append -p$.authors.[*].name -v "Fred Arminsen" -m
  • Ex: jsonconf path/to/conf.json append -p $.ids -v 113 -c int
• delete:
  • Deletes the specified endpoint, the path is optional here but if it's not passed, it deletes the whole document
  • Ex: jsonconf path/to/conf.json delete
  • Ex: jsonconf path/to/conf.json delete $.age
• edit:
  • Changes the value at the specified endpoint. Optionally convert the value to a nonstring type
  • Ex: jsonconf path/to/conf.json edit -p $.age -v 22 -c int
• shell:
  • Drops you into an interactive prompt working on the specified JSON file
  • Ex: jsonconf path/to/conf.json shell

Flags

These are the flags used by the command line interface

• -p/--path: The path flag, this is a JSONpath path that the script will use to walk the JSON representation.
• -o/--other: Used with the addfile command to specify a second JSON file to update the current on with.
• -v/--value: Used to denote the value or values being passed to the underlying command.
• -m/--multi: Used with the append command to specify that multiple paths should be written to, defaults to False.
• -c/--convert: A string used to denote how to convert the passed in value to it's final form. Acceptable values to pass to -c are any of str, bool, int, float, list, dict. You can also compose these values into a composite type as simple or complex as needed.

Interactive Prompt

Using the prompt is largely the same as the CLI. All the commands and flags are the same, however, all actions are applied to the file specified when the prompt was launched. There is also the write command avaliable (which is also technically available on the CLI, too, but the CLI autosaves) to save the current status of the document.

About -c/--convert and -v/--value

-c takes a space delimited string and uses it to build a converter out of it. This converter can be as simple as just int or as complex as dict int dict int list which would be a dictionary with integer keys and values that are sub dictionaries with integer keys and lists for values. Of course with increasingly complex datatypes comes increasing complicated input. For that last example, the input might look like: 4=4=value Which is confusing to look at until you break it down.

• -c list will convert a space delimited string into a list of values using Python's shlex.split function. This is smart enough to know that values encased in subquotes are to be considered one value. -c list -v "1 2 '3 4'" would output ['1', '2', '3 4'] in Python.
• -c dict will convert a space delimited string into sub strings delimited by = and then into dictionaries. -c dict -v "key=value other=something" creates {"key":"value", "other":"something"} inside Python.
• There is also -c bool which will return True for everything other than False (regardless of case), 0 and 0.0

The real power lies in stringing together the data types. If you need a list of integers, you'd specify -c "list int". If you need a dictionary with integer keys and list values, you'd specify -c "dict int list". But -c is also pretty smart, if you need a dictionary with sub dictionary values, you might think, -c "dict str dict" which would work, but you can simply use -c "dict dict" because -c knows that lists and dictionaries can't be used as keys, it doesn't even bother. It's also smart enough to know what -c wut means so it returns strings and that -c int float doesn't make sense so it just returns strings. Actually, I might alias -c to --PHP in the next release because when faced with something that maybe throwing an error is a good idea, it'll return strings.
The basic grammar of -c is this:

• Do I know what all the types are? If no, return string handler. Else:
• If there's just one type, return it. Else:
• If the first type isn't a list or dict, return a string handler because -c doesn't know what to do. Else:
  • If the first type is a list, build a secondary converter from the remaining parts of the original converter. Else:
  • If the frist type is a dict and there's only one following type or the next type is a list or dict, build a converter that will only manipulate the values of the dictionary from the rest of the original converter. Else:
  • If the first type is a dict and there's multiple remaining types and the next one is not a list or dict, build a converter that will manipulate both the key and the value from the remaining types. The next immediate type becomes the key type and the remaining types become the value type.

As you can imagine, this is recursive so something like dict int list float is easily constructed and the value can be passed as -v "0='1 2 3'" since Python's sh.lex intelligently groups matching quotes from the outside in.

Bugs and explosions

Due to the way -c is parsed it can be buggy at times, if you see something, say something. A copy of your conversion list and your value entry would be extremely helpful as I can't cover all possible permutations someone might use.
-c also currently makes no qualms about blowing up if you try to feed 3 4 through int either.

Interacting through Python

You can also import the module into your script to interact with it. The conversion sequences aren't really needed here since you can specify the exact type you'd like use instead of having to bend your mind around converting a flat value into nested dictionaries. The biggest tools here are JSONConfigParser and the commands module. The commands are largely the same, although in the Python API addfile and addfield become add_file and add_field respectively.

Adding custom types

Add custom scalar types (most likely scalar type transformations) is as easy as importing jsonconfigparser.utils.fieldtypes and updating the dictionary with your own mapping. For example, if you needed to have every value in a list upper case, you might add {'upper' : str.upper}. You can also override any existing converters as well as long as you support the expected input (or don't, that's your call). Of course, you'll need to ensure that your type modifications get added to the runtime of the shell prompt.

Adding custom commands

Adding custom commands is as simple as decoratoring a function with jsonconfigparser.command. Currently, there's not an easy way to add arguments to the command line and shell prompt and the decorator scans the function argument signature for named arguments (i.e. everything that's not *args/**kwargs) and the calling function attempts to pull them from the argparser object (or simple storage object for the prompt). Both of these need to be extended and possibly combined so adding arguments is easy as well.

Questions, Comments and Criticisms

Are always welcome. My email is available here. There's the github issue tracker. I'm not hard to track down, despite what some may lead you to believe. Let me know what you think about it all.