JSON-XS - decoding from json to a perl hash?

Posted on Sat Sep 6 21:24:35 2008 by keyslapper
decoding from json to a perl hash?
Hi all,

I'm having trouble understanding some of the methods in the JSON::XS module. I'm trying to pull a large JSON doc into nested perl structures - mostly in the form of an array of other JSON objects. The nesting goes down several layers, but probably not more than 10 or 12 levels.

The JSON document is currently over 400K, broken up into 5 top level objects. The vast majority of the data is in one of the top layer objects.

I'm having no trouble getting the data, but all I get is a text string that I can't seem to get into an array of hashes, or even one hash at a time. I'm trying to use a loop like the one in the sample code, and it is definitely reading everything.

It's not clear why the initial '[' and last ']' are to be removed. It's also not clear whether that text before the first '[' and after the last ']' is to be removed as well, though it seems it should.

The ultimate goal is to get a JSON object mapped to a Perl hash, where at least one of the hash members maps to an array of hashes of (roughly) the same type.

As I understand it, I need to do the following:
read data (I'm reading 65535 bytes at a time)
call $json->incr_parse on this data with a null context
remove leading white space if any
call $json->incr_parse with a scalar or list context
act on the object

Now, when I look at the text (prior to parsing), there is no leading '[', but there is a leading '{' (and a trailing '}') encompassing the entire JSON document. When I dump the object after the initial incremental parse, it looks like encoded text. I had thought it should be a reference to a hash, containing the first object (which is actually a root object encompassing the full JSON doc). When I try to pass this object to a subroutine as a reference to a hash, it doesn't appear I'm getting a valid object reference.

I'm checking it as follows:
${$_}{"type"} or die "no valid child structure.";
Every JSON object in this doc will have a type key, so I'm guessing I'm mistaking the context under which I should be accessing the object returned by $json->inc_parse.
The error I'm getting now is:
malformed JSON string, neither array, object, number, string or atom, at character offset 0 ["x]')) +;%20var%20submi..."] at ...
Here's the code in that section:
read STDIN, my $buf, 65535 or die "read error: $!"; $json->incr_parse ($buf); # void context, so no parsing $json->incr_text =~ s/^.*\[//x; for (;;) { for (;;) { my $obj = $json->incr_parse; # <-- XXX This is where the error happens XXX last unless $obj; # do something with $obj process_object($objref); } last if eof(); # add more data read STDIN, my $buf, 65535 or die "read error: $!"; $json->incr_parse ($buf); # void context, so no parsing }
Some of this is borrowed from the example in the online documentation

Further down, in the mapping section, it says that a JSON object will become a reference to a hash, so that's how I'm handling it in my subroutine, but the json error above happens at the incremental parse.

I'm not really a Perl beginner, but I'm a bit "rusty" as it were, so it's likely there's some simple Perlism I'm forgetting. Any suggestions of how to resolve this would be appreciated.
Write a response