Object-InsideOut - Re: significant OIO overhead with thousands of objects

Posted on Thu Nov 22 22:02:04 2007 by shmem in response to 6475 (See the whole thread of 13)
Re: significant OIO overhead with thousands of objects

As an immediate relieve, check out Anno Siegel's Alter package which provides Inside-Out functionality at a bit more than no cost.

The OIO engine is quite heavy, so I'm not surprised by the overhead per se, but by the amount. I made tests with a 50.001 lines file, each line consisting of an int and 52 more fields, as (a..z) alternated with random words from/usr/share/dict/words.

perl -nle 'chop && push@w, $_ if !($.%10) } \ { $w=@w;print join("\t",$_,map{$_,$w[rand$w-1]}a..z) for 0..50000' \ /usr/share/dict/words > text

Well, not 50.001, but 49850, since some values in the a..z hash are undefined. Anyways...

Built OIO-object from each line, then Alter-objects:

perl -MWords -nle 'use Time::HiRes qw(gettimeofday tv_interval);BEGIN{$tz=[gettimeofday]} @l=split/ +\t/,$_;push @o,Words->new("uid",@l) if @l == 53}{print tv_interval($tz,[gettimeofday])' text 29.586015 perl -MWords2 -nle 'use Time::HiRes qw(gettimeofday tv_interval);BEGIN{$tz=[gettimeofday]} @l=split +/\t/,$_;push @o,Words2->new("uid",@l) if @l == 53}{print tv_interval($tz,[gettimeofday])' text 7.258618

Some improvement. I don't know what your constructor does, here are the two packages I used for the test:

package Words; use Object::InsideOut; my @uid :Field :Accessor(uid); my @a :Field :Accessor(a); my @b :Field :Accessor(b); my @c :Field :Accessor(c); my @d :Field :Accessor(d); my @e :Field :Accessor(e); my @f :Field :Accessor(f); my @g :Field :Accessor(g); my @h :Field :Accessor(h); my @i :Field :Accessor(i); my @j :Field :Accessor(j); my @k :Field :Accessor(k); my @l :Field :Accessor(l); my @m :Field :Accessor(m); my @n :Field :Accessor(n); my @o :Field :Accessor(o); my @p :Field :Accessor(p); my @q :Field :Accessor(q); my @r :Field :Accessor(r); my @s :Field :Accessor(s); my @t :Field :Accessor(t); my @u :Field :Accessor(u); my @v :Field :Accessor(v); my @w :Field :Accessor(w); my @x :Field :Accessor(x); my @y :Field :Accessor(y); my @z :Field :Accessor(z); my %init_args :InitArgs = ( map { $_, { Mandatory => 1 } } a..z ); sub init :Init { my ($self, $args) = @_; #no strict 'refs'; $self->$_($args->{$_}) for keys %$args; }; package Words2; use Alter ego => {}; sub new { my $class = shift; my $self = bless( \ my $o, $class); my $href = ego( $self); %$href = @_; $self; } # retrieve values sub AUTOLOAD { my $self = shift; (my $key = $AUTOLOAD) =~ s/.*:://; ego( $self) -> { $key}; }; 1;

What perl version do you have? what version of Scalar::Utils? Plain perl or XS version?

Write a response