IB 7.5, running on Fedora 9
Backing up a 25G database resulted in an backup file of less than 3G. No
error message. Restoring this backup file resulted in a 5G database. Again
no error messages, and the new database worked all right. Since I recieved
no error messages during backup and restore, and since I had deleted most of
the text log records (BLOB), which are produced during data import, I
ignored the 25G versus 5G warning. I was stupid and deleted the original
25G database.
I later discovered that the biggest table, T_PERIODEDATA, only had 20E6
records left after restoring. The table T_PERIODEDATA contains hourly
consumption of energy, gas, current etc, collected from approx. 15000 meters
in different buildings in Norway.
This is the metadata for T_PERIODEDATA, with size in bytes for each column:
CREATE TABLE "T_PERIODEDATA"
(
"PD" NUMERIC(18, 0) NOT NULL, /* 8 bytes */
"PD_PG" INTEGER NOT NULL, /* 4 bytes */
"PD_VERSJON" SMALLINT NOT NULL, /* 8 bytes */
"PD_TYPE" CHAR(1) NOT NULL, /* 1 bytes */
"PD_FOM" TIMESTAMP NOT NULL, /* 8 bytes */
"PD_OT" TIMESTAMP NOT NULL, /* 8 bytes */
"PD_FORBRUK" NUMERIC(18, 4) NOT NULL, /* 8 bytes */
"PD_STAND" NUMERIC(18, 4) NOT NULL, /* 8 bytes */
"PD_TOTAL" NUMERIC(18, 4) NOT NULL, /* 8 bytes */
"PD_CRC" INTEGER NOT NULL, /* 4 bytes */
CONSTRAINT "PK_PERIODEDATA" PRIMARY KEY ("PD"),
CONSTRAINT "UQ_PD_FOM" UNIQUE ("PD_PG", "PD_FOM")
);
ALTER TABLE "T_PERIODEDATA" ADD CONSTRAINT "FK_PD_PG"
FOREIGN KEY ("PD_PG")
REFERENCES "T_PULSGIVER" ("PG") ON UPDATE CASCADE ON DELETE
CASCADE;
Refferring to http://www.ib-aid.com/interbase/firebird/bug/research.html
Adding header of 14 bytes, the record size of T_PERIODEDATA is 73 bytes.
Maximum page count for one table can be calculated as MaxDataPageCount =
(MaxInt / PageSize) * 17.476 .
With page size = 4096, one page can contain 4096/73 = 56 records. Therefore
the record limit of T_PERIODEDATA is approx. 500E6 records, .
If record limit is exceeded I should find "pointer page vanished from
DPM_next (249)" in the interbase.log and my database should stop working.
This has not happend.
There is another table containg information about daily consumption. This
table, T_DOGN_FORBRUK, has 8584276 records with no corresponding records in
T_PERIODEDATA. This means that T_PERIODEDATA has lost 8584276 * 24 = 206E6
records. This is only half of the calculated record limit of 500E6. This
also means that the T_DOGN_FORBRUK and T_PERIODEDATA tables are out of sync.
That should not happen.
We also know for certain that a backup < 3G (again the crazy size) was
produced automatically the 20th october, and that invoice produced (Based on
T_PERIODEDATA) the 23th of october was succesfull. After my manually backup
and restore the 24th october, the data in T_PERIODEDATA used for the invoice
at 23th october are partly missing.
The automatic backup is done on a windows server, while tha manual backup is
done on the Linux (Fedora 9) server.
I am certain the problem is caused by gbak.
Is this a known issue?
Jarle Nilsen
Scandinavian Electric AS
Dir .telf :55 50 60 41
Fax: 55 50 60 99
Mobil 40 40 21 58
E-post: jarle.nilsen@scel.no
Besøk vår hjemmeside: http://www.scel.no
|
gbak doesn't use 32 bit I/O and doesn't need to. It simply appends to
the file for backup and reads serially through it for restore.
Therefore, the only limitations on file size for gbak are imposed by the
OS.
As Bill has suggested, gbak loosing data implies a corrupt database as
gbak reads data from the database just as any other client does.
Bill Todd wrote:
> <jarle.nilsen@hjemme.no> wrote:
>
>> Could there be an 8G limit for backup files?
>
> No. File system size limits are determined by whether they use a 32 bit
> signed integer for the file size (max = 2g), an unsigned 32 bit integer
> (max = 4g) or a 64 bit integer (max = 18 terabytes).
>
> Before your try the multi-file backup you might want to run gfix
> -no_update and see if it reports any errors.
|