Lua unpack bug?

Tag: lua , unpack Author: rakun Date: 2009-10-19

I Have stumbled on a weird behavior in Lua unpack function

table1 = {true, nil, true, false, nil, true, nil}
table2 = {true, false, nil, false, nil, true, nil}

a1,b1,c1,d1,e1,f1,g1 = unpack( table1 )
print ("table1:",a1,b1,c1,d1,e1,f1,g1)

a2,b2,c2,d2,e2,f2,g2 = unpack( table2 )
print ("table2:",a2,b2,c2,d2,e2,f2,g2)

Output:

table1: true	nil	true	false	nil	nil	nil
table2: true	false	nil	nil	nil	nil	nil

The second unpack delivers parameters up to the first nil value. I could live with that. The first table delivers 4? parameters with one being nil in the middle. It has 4 parameters that are not nil, but they aren't the one that are shown.

Could anyone explain this? This was tried with codepad.org and lua 5.1

Best Answer

The problem can be resolved simply by specifying the beginning and ending indexes to unpack() and using the table.maxn() as the ending index:

table1 = {true, nil, true, false, nil, true, nil}

a1,b1,c1,d1,e1,f1,g1 = unpack( table1, 1, table.maxn(table1) )
print ("table1:",a1,b1,c1,d1,e1,f1,g1)
-->table1: true    nil     true    false   nil     true    nil

The true reason for the discrepancy on how the two tables are handled is in the logic of determining the length of the array portion of the table.

The luaB_unpack() function uses luaL_getn() which is defined in terms of lua_objlen() which calls luaH_getn() for tables. The luaH_getn() looks at the last position of the array, and if it is nil performs a binary search for a boundary in the table ("such that t[i] is non-nil and t[i+1] is nil"). The binary search for the end of the array is the reason that table1 is handled differently then table2.

This should only be an issue if the last entry in the array is nil.

From Programming in Lua (pg.16) (You should buy this book.): When an array has holes--nil elements inside it--the length operator may assume any of these nil elements as the end marker. Therefore, you should avoid using the length operator on arrays that may contain holes.

The unpack() is using the length operator lua_objlen(), which "may assume any of [the] nil elements as the end" of the array.

comments:

Thank you, you saved my day. Does this mean that table.maxn() traverses the whole allocated size for the table?
table.maxn() "Returns the largest positive numerical index of the given table" see lua.org/manual/5.1/manual.html#pdf-table.maxn
is table.maxn really guaranteed to return the correct value (= the number of objects in the table literal) here?
@kazer.se table.maxn will return 6 which is the largest index for the table table1 with a non-nil entry. The nil in the 7th position will not be counted by table.maxn; however, for this case it works perfectly. The unpack will return 6 elements and the 7th variable being assigned will receive nil as "the list is extended with as many nil's as needed" for the assignment.

Other Answer1

2.2 - Values and Types

[...] The type table implements associative arrays, that is, arrays that can be indexed not only with numbers, but with any value (except nil). Tables can be heterogeneous; that is, they can contain values of all types (except nil). [...]

Given nil to an entry will break the table enumeration and your variables wont be init properly.

Here is a simple example that demonstrates a problematic behavior:

table1 = {true, false, nil, false, nil, true, nil}
for k,v in ipairs(table1) do
  print(k, v)
end

output:

1   true
2   false
>Exit code: 0

comments:

More specifically, i suspect that it ignores the nils he specified.
No, it does not RCIX, in that case table 2 would have returned 4 values as well i guess.
The only reason that the for loop stopped is because the iterator returned nil at position 3. The table was still initialized properly, because assert(table1[1]==true and table1[2]==false and table1[3]==nil and table1[4]==false and table1[5]==nil and table1[6]==true and table1[7]==nil) does not assert.
@gwell, I said that the nil can break the enumeration. I didn't say that the elements will cease to exist.