INDEXBYTES

In the on-line help for SET IBLOCK, there is the following:

-----------------------------------------------------------
Changes the default block size used for new .MDX files.

Syntax

SET IBLOCK TO <expN>

<expN>

A number from 1 to 63 that sets the size of index blocks allocated to 
new .MDX files. The default value is 1. (The actual size in bytes is the 
number you specify multiplied by 512 bytes; however, the minimum size of 
a block is 1024 bytes.) To change the default, update the IBLOCK setting 
in PLUS.ini. To do so, either use the SET command to specify the setting 
interactively, or enter the IBLOCK parameter directly in PLUS.ini.

Description

Use SET IBLOCK to change the size of blocks in which dBASE Plus stores 
.MDX files on disk to improve the performance and efficiency of indexes. 
You can specify a block size from 1024 bytes to approximately 32K. The 
IBLOCK setting overrides any previous block size defined by the SET 
BLOCKSIZE command or specified in the PLUS.ini file. After the block 
size has been changed, new .MDX index files are created with the new 
block size.

Multiple index (.MDX) files are composed of individual index blocks (or 
nodes). Nodes contain the value of keys corresponding to individual 
records and provide the information to locate the appropriate record for 
each key value. Since the IBLOCK setting determines the size of nodes, 
the setting also determines the number of key values that can fit into 
each node. When a single node can't contain all the key values in an 
index, dBASE Plus creates one or more parent nodes. These intermediate 
nodes also contain key values. Instead of pointing to record numbers, 
however, intermediate nodes point to leaf nodes or other lower-level 
intermediate nodes. If you increase the size of index blocks and create 
a new .MDX file, the new and larger leaf nodes contain more key values.

Whether you can improve performance by storing key values in larger or 
smaller nodes depends on several factors: the distribution of data, if 
tables are linked together, the length of key values, the value of 
INDEXBYTES, and the type of operation requested. Typically, every .MDX 
file contains more than one index tag. Finding the best setting for a 
given .MDX file requires experimentation because the best size for one 
index tag might not be the best size for another.

The following is a list of basic principles governing index performance.

Since nodes might not be sequential, dBASE Plus reads only one node at a 
time from the disk. Reading more than one node is usually inefficient, 
because typically the second node is not the next node in the sequential 
list.

Once a node is read into memory, dBASE Plus attempts to store it there 
for later use. The amount of space devoted to caching the index nodes is 
determined by the setting of INDEXBYTES.
	When users link several tables together, for example, with SET 
RELATION, performance is better if all the relevant nodes for the tables 
are in memory simultaneously. For example, if a large node for table B 
pushes out the previously read node for table A, dBASE Plus must find 
and read the table A node again from disk when the node for table A 
needs to be used again. If both nodes remain in memory, performance can 
be improved.

When tables have many identical key values, dBASE Plus might have to 
store them in many nodes. In this situation, performance might be 
improved by increasing the node size so that dBASE Plus reads fewer 
nodes from disk to load the same number of key values into memory.
	Small node sizes can cause performance degradation. This occurs 
because as nodes are read in and out, dBASE Plus attempts to cache them 
all. When the small nodes are removed from memory by more recently read 
nodes, they leave unused spaces in memory that are too small to contain 
larger nodes. Over time, memory can become fragmented, resulting in 
slower performance.
-----------------------------------------------------------


You will notice it mentions several times "INDEXBYTES". Yet INDEXBYTES

is not documented anywhere. (That's the bug).


Until the next release of the on-help comes out with this documented, 
would it be possible to learn:

(1) How/where to set INDEXBYTES?
(2) The range of valid values?
(3) The current default value?

Any recommendations or caveats would be welcome.

Many thanks in advance!


-- 
Geoff Wass [dBVIPS]
Montréal, Québec, Canada

.|.|.|        dBASE info at http://geocities.com/geoff_wass       |.|.|.
.|.|.| ---------------------------------------------------------- |.|.|.

Hi Geoff,

> You will notice it mentions several times "INDEXBYTES". Yet
INDEXBYTES
> is not documented anywhere. (That's the bug).
>
>
> Until the next release of the on-help comes out with this documented,
> would it be possible to learn:
>
> (1) How/where to set INDEXBYTES?
> (2) The range of valid values?
> (3) The current default value?
>
> Any recommendations or caveats would be welcome.
>
> Many thanks in advance!


I used Google to search for INDEXBYTES and found the following info (at your 
website):

http://www.geocities.com/geoff_wass/dBASE/GaryWhite/dBASE/FAQ/qdbmem.htm

////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////
INDEXBYTES = <number>
Range 2 - 2048
Default 256
Each record in a database file is represented by a node in an index file. 
Each node contains the key expression value for its associated record and a 
reference to the next node in the index. As you access an index file, its 
nodes are loaded into memory as needed. dBASE IV loads the nodes into memory 
in groups determined by the BLOCKSIZE setting when the index file was 
created; the default BLOCKSIZE is 512K bytes. The number of index file 
blocks that can be in memory at one time is determined by the INDEXBYTES 
value. If you have large and complex index key expressions, you may benefit 
by increasing the INDEXBYTES setting. By setting INDEXBYTES to a high value, 
you can load more nodes into memory at one time and thereby increase your 
index access time. However, if you have large database files with short 
index key expressions, you may be wasting time by unnecessarily loading 
nodes into memory as you traverse the index. Decrease the value of 
INDEXBYTES if you frequently get the error message Insufficient memory.

////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////

As far as I can determine, INDEXBYTES is a parameter that can be set in 
config.db for dBASE IV (and maybe dBASE 5 for DOS).
It appears to have been superceded (or obsoleted) by, first the 16 bit BDE 
in dBASE for Windows and later, by the 32 bit BDE starting with Visual 
dBASE.  The BDE does not appear to have any setting that is equivalent to 
INDEXBYTES.

We are reviewing the SET IBLOCK help topic to see about removing the 
references to INDEXBYTES.

Thanks,

- Marty

Martin Kay
dataBased Intelligence, Inc.

In article <34qkehLdIHA.1880@news-server>, mkay@databi.com says...
> Hi Geoff,
> 
> > You will notice it mentions several times "INDEXBYTES". Yet
INDEXBYTES
> > is not documented anywhere. (That's the bug).
> >
> >
> > Until the next release of the on-help comes out with this documented,
> > would it be possible to learn:
> >
> > (1) How/where to set INDEXBYTES?
> > (2) The range of valid values?
> > (3) The current default value?
> >
> > Any recommendations or caveats would be welcome.
> >
> > Many thanks in advance!
> 
> 
> I used Google to search for INDEXBYTES and found the following info (at
your 
> website):
> 
> http://www.geocities.com/geoff_wass/dBASE/GaryWhite/dBASE/FAQ/qdbmem.htm
> 
>
////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////
> INDEXBYTES = <number>
> Range 2 - 2048
> Default 256
> Each record in a database file is represented by a node in an index file. 
> Each node contains the key expression value for its associated record and a

> reference to the next node in the index. As you access an index file, its 
> nodes are loaded into memory as needed. dBASE IV loads the nodes into
memory 
> in groups determined by the BLOCKSIZE setting when the index file was 
> created; the default BLOCKSIZE is 512K bytes. The number of index file 
> blocks that can be in memory at one time is determined by the INDEXBYTES 
> value. If you have large and complex index key expressions, you may benefit

> by increasing the INDEXBYTES setting. By setting INDEXBYTES to a high
value, 
> you can load more nodes into memory at one time and thereby increase your 
> index access time. However, if you have large database files with short 
> index key expressions, you may be wasting time by unnecessarily loading 
> nodes into memory as you traverse the index. Decrease the value of 
> INDEXBYTES if you frequently get the error message Insufficient memory.
> 
>
////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////
> 
> As far as I can determine, INDEXBYTES is a parameter that can be set in 
> config.db for dBASE IV (and maybe dBASE 5 for DOS).
> It appears to have been superceded (or obsoleted) by, first the 16 bit BDE

> in dBASE for Windows and later, by the 32 bit BDE starting with Visual 
> dBASE.  The BDE does not appear to have any setting that is equivalent to 
> INDEXBYTES.
> 
> We are reviewing the SET IBLOCK help topic to see about removing the 
> references to INDEXBYTES.
> 
> Thanks,
> 
> - Marty
> 
> Martin Kay
> dataBased Intelligence, Inc.

 

Marty,

Yes, I am aware of the DOS documentation and the contents more than 
piqued my interest.

If there is no equivalent to INDEXBYTES in the 32-bit product, can I be 
put down for a wish? Being able to assign more memory to index blocks 
could lead to improvements in speed, wouldn't it? Or am I expecting too 
much? (Yeah, I know...)

-- 
Geoff Wass [dBVIPS]
Montréal, Québec, Canada

.|.|.|        dBASE info at http://geocities.com/geoff_wass       |.|.|.
.|.|.| ---------------------------------------------------------- |.|.|.

Geoff,

> If there is no equivalent to INDEXBYTES in the 32-bit product, can I be
> put down for a wish? Being able to assign more memory to index blocks
> could lead to improvements in speed, wouldn't it? Or am I expecting too
> much? (Yeah, I know...)

I added this as QAID: 6381 - with the title: Add ability to optimize index 
search speeds

- Marty -

Martin Kay
dataBased Intelligence, Inc.

In article <$PwyveXdIHA.1872@news-server>, mkay@databi.com says...
> Geoff,
> 
> > If there is no equivalent to INDEXBYTES in the 32-bit product, can I
be
> > put down for a wish? Being able to assign more memory to index blocks
> > could lead to improvements in speed, wouldn't it? Or am I expecting
too
> > much? (Yeah, I know...)
> 
> I added this as QAID: 6381 - with the title: Add ability to optimize index

> search speeds
> 
> - Marty -
> 
> Martin Kay
> dataBased Intelligence, Inc.


Marty,

Many thanks for that! Have a good weekend.

-- 
Geoff Wass [dBVIPS]
Montréal, Québec, Canada

.|.|.|        dBASE info at http://geocities.com/geoff_wass       |.|.|.
.|.|.| ---------------------------------------------------------- |.|.|.