Discussion:
Indexes on VFP tables--when do they get updated
m***@mbsoftwaresolutions.com
2018-01-24 18:29:09 UTC
Permalink
Question for you legends of the Fox (data):

Say Table1 has "X" number of indexes. Program code does something like
the following:

IF Condition1 THEN
REPLACE Field1 with SomeValue in Table1
ENDIF

IF Condition2 THEN
REPLACE Field48 with Datetime() in Table1
ENDIF

Are each of the "X" indexes updated for each replace, even if those
indexes do not include any reference to Field1 or Field48, or only
indexes that are affected by the data value change?

I understand that ADDing and DELETEing records updated all indexes; I
would think the UPDATEs--regardless of what fields updated--would cause
index updates for every indexed field on every update as well.

tia,
--Michael

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Kurt at VR-FX
2018-01-24 18:48:04 UTC
Permalink
Mike,

If I was to look at this from a totally Logical stand point - I would
think the Indexes are ONLY update if you update a field in a record
where that field is contained in one or more of the Indices.

However, Why not do a simple test - and generate your Own answer!??

Take a chunk of the data from your table in question - or even take a
copy of the Whole thing, along with the Indices, drop into another
folder. Then - simply do some updates based upon your criteria below -
and then you can confirm Exactly what happens!

Right???

:-)

-K-
Say Table1 has "X" number of indexes.  Program code does something
IF Condition1 THEN
  REPLACE Field1 with SomeValue in Table1
ENDIF
IF Condition2 THEN
  REPLACE Field48 with Datetime() in Table1
ENDIF
Are each of the "X" indexes updated for each replace, even if those
indexes do not include any reference to Field1 or Field48, or only
indexes that are affected by the data value change?
I understand that ADDing and DELETEing records updated all indexes; I
would think the UPDATEs--regardless of what fields updated--would
cause index updates for every indexed field on every update as well.
tia,
--Michael
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/9ab07317-d4fe-6c03-cba0-***@optonline.net
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who ar
m***@mbsoftwaresolutions.com
2018-01-24 18:58:27 UTC
Permalink
Post by Kurt at VR-FX
Mike,
If I was to look at this from a totally Logical stand point - I would
think the Indexes are ONLY update if you update a field in a record
where that field is contained in one or more of the Indices.
However, Why not do a simple test - and generate your Own answer!??
Take a chunk of the data from your table in question - or even take a
copy of the Whole thing, along with the Indices, drop into another
folder. Then - simply do some updates based upon your criteria below -
and then you can confirm Exactly what happens!
Right???
:-)
-K-
But if I did that, think of the discussion and thread that would be
missed here. LOL

This question relates to VFP CDX files but is something in theory too
for other systems (although I realize they may do things differently).

I just dropped a boatload of nonsense indexes from a client's legacy app
database and believe that it (theoretically) should improve network
traffic by lessening these unneeded updates.

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Paul H. Tarver
2018-01-24 19:18:16 UTC
Permalink
It would only be missed if you didn't report your results and included a
spreadsheet and a graph! :)

Paul


-----Original Message-----
From: ProfoxTech [mailto:profoxtech-***@leafe.com] On Behalf Of
***@mbsoftwaresolutions.com
Sent: Wednesday, January 24, 2018 12:58 PM
To: ***@leafe.com
Subject: Re: Indexes on VFP tables--when do they get updated
Post by Kurt at VR-FX
Mike,
If I was to look at this from a totally Logical stand point - I would
think the Indexes are ONLY update if you update a field in a record
where that field is contained in one or more of the Indices.
However, Why not do a simple test - and generate your Own answer!??
Take a chunk of the data from your table in question - or even take a
copy of the Whole thing, along with the Indices, drop into another
folder. Then - simply do some updates based upon your criteria below -
and then you can confirm Exactly what happens!
Right???
:-)
-K-
But if I did that, think of the discussion and thread that would be missed
here. LOL

This question relates to VFP CDX files but is something in theory too for
other systems (although I realize they may do things differently).

I just dropped a boatload of nonsense indexes from a client's legacy app
database and believe that it (theoretically) should improve network traffic
by lessening these unneeded updates.

[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/023701d39548$1698e860$43cab920$@tpcqpc.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Alan Bourke
2018-01-24 19:35:08 UTC
Permalink
Only the indexes that involve a 'touched' field. And even then it's just a small update.
--
Alan Bourke
alanpbourke (at) fastmail (dot) fm
Post by m***@mbsoftwaresolutions.com
Say Table1 has "X" number of indexes. Program code does something like
IF Condition1 THEN
REPLACE Field1 with SomeValue in Table1
ENDIF
IF Condition2 THEN
REPLACE Field48 with Datetime() in Table1
ENDIF
Are each of the "X" indexes updated for each replace, even if those
indexes do not include any reference to Field1 or Field48, or only
indexes that are affected by the data value change?
I understand that ADDing and DELETEing records updated all indexes; I
would think the UPDATEs--regardless of what fields updated--would cause
index updates for every indexed field on every update as well.
tia,
--Michael
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@webmail.messagingengine.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
m***@mbsoftwaresolutions.com
2018-01-24 20:11:15 UTC
Permalink
Thanks. In this app, they indexed EVERY field, and the table in
question had a CDX size of 233 MB before I pruned it. Now it's down to
56 MB.
Post by Alan Bourke
Only the indexes that involve a 'touched' field. And even then it's just a small update.
--
Alan Bourke
alanpbourke (at) fastmail (dot) fm
On Wed, 24 Jan 2018, at 6:29 PM,
Post by m***@mbsoftwaresolutions.com
Say Table1 has "X" number of indexes. Program code does something like
IF Condition1 THEN
REPLACE Field1 with SomeValue in Table1
ENDIF
IF Condition2 THEN
REPLACE Field48 with Datetime() in Table1
ENDIF
Are each of the "X" indexes updated for each replace, even if those
indexes do not include any reference to Field1 or Field48, or only
indexes that are affected by the data value change?
I understand that ADDing and DELETEing records updated all indexes; I
would think the UPDATEs--regardless of what fields updated--would cause
index updates for every indexed field on every update as well.
tia,
--Michael
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Stephen Russell
2018-01-24 20:19:00 UTC
Permalink
I expect the update of .cdx to be at the same time that the .dbf is done.

223 meg is a tiny index size and reducing it down to 1/4 probably didn't
make anything faster, did it? Now an index of 20 gig that would probably
be noticed if you brought it in line to only 5 gig.



On Wed, Jan 24, 2018 at 2:11 PM, <
Thanks. In this app, they indexed EVERY field, and the table in question
had a CDX size of 233 MB before I pruned it. Now it's down to 56 MB.
Post by Alan Bourke
Only the indexes that involve a 'touched' field. And even then it's just a small update.
--
Alan Bourke
alanpbourke (at) fastmail (dot) fm
On Wed, 24 Jan 2018, at 6:29 PM,
Post by m***@mbsoftwaresolutions.com
Say Table1 has "X" number of indexes. Program code does something like
IF Condition1 THEN
REPLACE Field1 with SomeValue in Table1
ENDIF
IF Condition2 THEN
REPLACE Field48 with Datetime() in Table1
ENDIF
Are each of the "X" indexes updated for each replace, even if those
indexes do not include any reference to Field1 or Field48, or only
indexes that are affected by the data value change?
I understand that ADDing and DELETEing records updated all indexes; I
would think the UPDATEs--regardless of what fields updated--would cause
index updates for every indexed field on every update as well.
tia,
--Michael
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAJidMYJAuJudu5xfDem+MJsAHRA2NOOb4Gb7=***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
m***@mbsoftwaresolutions.com
2018-01-24 22:53:47 UTC
Permalink
Post by Stephen Russell
I expect the update of .cdx to be at the same time that the .dbf is done.
223 meg is a tiny index size and reducing it down to 1/4 probably didn't
make anything faster, did it? Now an index of 20 gig that would probably
be noticed if you brought it in line to only 5 gig.
VFP files can't be more than 2 GB, Stephen.

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Stephen Russell
2018-01-24 23:07:15 UTC
Permalink
I thought that the dbf had that limit. What if you had 25 indexes on a 50
column table. You would probably blow out the .cdx space before the dbf.



On Wed, Jan 24, 2018 at 4:53 PM, <
Post by m***@mbsoftwaresolutions.com
Post by Stephen Russell
I expect the update of .cdx to be at the same time that the .dbf is done.
223 meg is a tiny index size and reducing it down to 1/4 probably didn't
make anything faster, did it? Now an index of 20 gig that would probably
be noticed if you brought it in line to only 5 gig.
VFP files can't be more than 2 GB, Stephen.
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAJidMYKDa-wVxNFxeTdVHqFf55=ryA3ktvsD4TR0MEhZbV+***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Kurt at VR-FX
2018-01-25 04:25:38 UTC
Permalink
Forgive Stephen - as he now plays with the Big Boys like M$ SQL!

:-)

-K-
Post by m***@mbsoftwaresolutions.com
Post by Stephen Russell
I expect the update of .cdx to be at the same time that the .dbf is done.
223 meg is a tiny index size and reducing it down to 1/4 probably didn't
make anything faster, did it?  Now an index of  20 gig that would
probably
be noticed if you brought it in line to only 5 gig.
VFP files can't be more than 2 GB, Stephen.
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/50f879eb-6caf-1ddd-8d28-***@optonline.net
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to
Stephen Russell
2018-01-25 15:28:09 UTC
Permalink
I have some indexes larger than your last hard drive. :)
Table Name # Records Reserved (KB) Data (KB) Indexes (KB) Unused (KB)
dbo.ttfgld495600 95,953,505 251,547,152 138,105,072 111,772,128 1,669,952

dbo.ttfgld482600 105,142,512 146,331,720 100,144,320 45,854,776 332,624

dbo.ttfgld465600 100,724,986 135,428,712 134,551,896 859,840 16,976
dbo.ttfgld106600 85,175,413 78,139,496 46,399,096 31,710,944 29,456
Post by Kurt at VR-FX
Forgive Stephen - as he now plays with the Big Boys like M$ SQL!
:-)
-K-
Post by m***@mbsoftwaresolutions.com
Post by Stephen Russell
I expect the update of .cdx to be at the same time that the .dbf is done.
223 meg is a tiny index size and reducing it down to 1/4 probably didn't
make anything faster, did it? Now an index of 20 gig that would probably
be noticed if you brought it in line to only 5 gig.
VFP files can't be more than 2 GB, Stephen.
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAJidMYK23Q67NDjwxQ+niaZrc7P0QdrwrDuxtAd=***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Ted Roche
2018-01-24 20:21:19 UTC
Permalink
First off, don't do that. REPLACE (or better, UPDATE) once. Especially
in high network traffic situations, the amount of time it takes to
assemble a field and value list to issue one:

UPDATE TableName SET &lcSetFieldsToValues WHERE FilterCriteria

saves an enormous amount of I/O and CPU cycles: one lock, one
transaction, one row re-write, one set of reindexes.

Second, in VFP, all indexes have to be evaluated, as there's no
backlink to which fields are used in which expressions. Indexes are
defined at the table level, not the row, so can have multiple field
names, functions (UPPER() or DELETED(), for example), concatentations
or just random stuff.

<OldWarStory>
I once worked on an app where the original developer thought it would
be a "clever" idea to define index expressions as UDFs. Yes, it's
possible. In his UDFs, he would switch work areas, open tables if not
already opened, look up values, and then return the value, cleaning up
work areas and tables as he went. For every index definition for every
record. A reindex with more than a couple hundred records brought the
entire system to its knees.
</OldWarStory>

On Wed, Jan 24, 2018 at 1:29 PM,
I understand that ADDing and DELETEing records updated all indexes; I would
think the UPDATEs--regardless of what fields updated--would cause index
updates for every indexed field on every update as well.
I don't know if it actually caused *UPDATES* but I think VFP would
have to go through each index expression and test to see if the value
had changed. Perhaps they'd skip the write if it was unchanged.

Other database engines do this differently
--
Ted Roche
Ted Roche & Associates, LLC
http://www.tedroche.com

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
m***@mbsoftwaresolutions.com
2018-01-24 22:57:48 UTC
Permalink
Post by Ted Roche
First off, don't do that. REPLACE (or better, UPDATE) once. Especially
in high network traffic situations, the amount of time it takes to
UPDATE TableName SET &lcSetFieldsToValues WHERE FilterCriteria
saves an enormous amount of I/O and CPU cycles: one lock, one
transaction, one row re-write, one set of reindexes.
Yes, I retooled some code to replace (pun not intended) a whole bunch of
REPLACE commands in the same area with a SCATTER to object, do the
voodoo, then just ONE REPLACE afterwards for that very purpose. I
specifically listed that "multiple conditions" example thinking if I
didn't, someone would say as you did to avoid the multiple REPLACEs.
:-)
Post by Ted Roche
Second, in VFP, all indexes have to be evaluated, as there's no
backlink to which fields are used in which expressions. Indexes are
defined at the table level, not the row, so can have multiple field
names, functions (UPPER() or DELETED(), for example), concatentations
or just random stuff.
So you're contending that ALL of the indexes--not just those affected by
the field that was changed--would have to be reevaluated?
Post by Ted Roche
<OldWarStory>
I once worked on an app where the original developer thought it would
be a "clever" idea to define index expressions as UDFs. Yes, it's
possible. In his UDFs, he would switch work areas, open tables if not
already opened, look up values, and then return the value, cleaning up
work areas and tables as he went. For every index definition for every
record. A reindex with more than a couple hundred records brought the
entire system to its knees.
</OldWarStory>
YIKES!!!! Thanks for sharing that one!

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Fernando D. Bozzo
2018-01-24 23:39:27 UTC
Permalink
When the index is evaluated initially, VFP knows exactly what fields affect
what index, so really just affected indexes are updated.

It's easy to test it. just make an old index (IDX) on two different fields:

CREATE TABLE test (field1 C(10), field2 I)
INDEX on field1 TO test_f1 additive
INDEX on field2 TO test_f2 additive

Add some data, wait a minute, replace one of the fields and do a FLUSH
FORCE, and you will see that only the affected index changes his timestamp.
Post by m***@mbsoftwaresolutions.com
Post by Ted Roche
First off, don't do that. REPLACE (or better, UPDATE) once. Especially
in high network traffic situations, the amount of time it takes to
UPDATE TableName SET &lcSetFieldsToValues WHERE FilterCriteria
saves an enormous amount of I/O and CPU cycles: one lock, one
transaction, one row re-write, one set of reindexes.
Yes, I retooled some code to replace (pun not intended) a whole bunch of
REPLACE commands in the same area with a SCATTER to object, do the voodoo,
then just ONE REPLACE afterwards for that very purpose. I specifically
listed that "multiple conditions" example thinking if I didn't, someone
would say as you did to avoid the multiple REPLACEs. :-)
Post by Ted Roche
Second, in VFP, all indexes have to be evaluated, as there's no
backlink to which fields are used in which expressions. Indexes are
defined at the table level, not the row, so can have multiple field
names, functions (UPPER() or DELETED(), for example), concatentations
or just random stuff.
So you're contending that ALL of the indexes--not just those affected by
the field that was changed--would have to be reevaluated?
Post by Ted Roche
<OldWarStory>
I once worked on an app where the original developer thought it would
be a "clever" idea to define index expressions as UDFs. Yes, it's
possible. In his UDFs, he would switch work areas, open tables if not
already opened, look up values, and then return the value, cleaning up
work areas and tables as he went. For every index definition for every
record. A reindex with more than a couple hundred records brought the
entire system to its knees.
</OldWarStory>
YIKES!!!! Thanks for sharing that one!
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAGQ_Ju=4VNPRZZ5QXsK2iY9eY55thp4vpiO=***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
m***@mbsoftwaresolutions.com
2018-01-25 16:15:30 UTC
Permalink
Post by Fernando D. Bozzo
When the index is evaluated initially, VFP knows exactly what fields affect
what index, so really just affected indexes are updated.
CREATE TABLE test (field1 C(10), field2 I)
INDEX on field1 TO test_f1 additive
INDEX on field2 TO test_f2 additive
Add some data, wait a minute, replace one of the fields and do a FLUSH
FORCE, and you will see that only the affected index changes his timestamp.
I don't think that's a fair test to use IDX files since this example
uses a CDX file, and it's all in the same file.

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Kurt at VR-FX
2018-01-25 18:24:08 UTC
Permalink
IN theory  - the same test should be viable - just see if the CDX
Timestamp was updated!
Post by m***@mbsoftwaresolutions.com
Post by Fernando D. Bozzo
When the index is evaluated initially, VFP knows exactly what fields affect
what index, so really just affected indexes are updated.
CREATE TABLE test (field1 C(10), field2 I)
INDEX on field1 TO test_f1 additive
INDEX on field2 TO test_f2 additive
Add some data, wait a minute, replace one of the fields and do a FLUSH
FORCE, and you will see that only the affected index changes his timestamp.
I don't think that's a fair test to use IDX files since this example
uses a CDX file, and it's all in the same file.
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/fdb128bc-4d95-f140-2c31-***@optonline.net
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those l
m***@mbsoftwaresolutions.com
2018-01-25 18:47:24 UTC
Permalink
Post by Kurt at VR-FX
IN theory  - the same test should be viable - just see if the CDX
Timestamp was updated!
Yes, it would be, but my point is that the entire CDX would come over
the line, if I understand correctly. The entire DBF doesn't come over
the wire (network), but the CDX does?

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawye
Gene Wirchenko
2018-01-25 18:51:24 UTC
Permalink
Post by m***@mbsoftwaresolutions.com
Post by Fernando D. Bozzo
When the index is evaluated initially, VFP knows exactly what fields affect
what index, so really just affected indexes are updated.
CREATE TABLE test (field1 C(10), field2 I)
INDEX on field1 TO test_f1 additive
INDEX on field2 TO test_f2 additive
Add some data, wait a minute, replace one of the fields and do a FLUSH
FORCE, and you will see that only the affected index changes his timestamp.
I don't think that's a fair test to use IDX files since this example
uses a CDX file, and it's all in the same file.
I think it *is* a fair test for exactly that reason. If you
see only one of the IDX index files change, that suggests that VFP
only updates an index where it would make a difference.

I grant that it could be that that optimisation does not exist
in the CDX handling, but how likely do you think that is?

Sincerely,

Gene Wirchenko


_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mtlp000084
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
m***@mbsoftwaresolutions.com
2018-01-25 19:02:45 UTC
Permalink
Post by m***@mbsoftwaresolutions.com
Post by Fernando D. Bozzo
When the index is evaluated initially, VFP knows exactly what fields affect
what index, so really just affected indexes are updated.
CREATE TABLE test (field1 C(10), field2 I)
INDEX on field1 TO test_f1 additive
INDEX on field2 TO test_f2 additive
Add some data, wait a minute, replace one of the fields and do a FLUSH
FORCE, and you will see that only the affected index changes his timestamp.
I don't think that's a fair test to use IDX files since this example
uses a CDX file, and it's all in the same file.
I think it *is* a fair test for exactly that reason. If you see
only one of the IDX index files change, that suggests that VFP only
updates an index where it would make a difference.
I grant that it could be that that optimisation does not exist in
the CDX handling, but how likely do you think that is?
Perhaps a better question at this point: Does the CDX file get pulled
down locally for every query? I would think for ADD/UPDATE/DELETEs, it
would get updated on the LAN. (In this scenario, the database is on a
network folder; not local.)

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Richard Kaye
2018-01-25 19:05:59 UTC
Permalink
My memory is the CDX is sent as soon as you open the table along with the header from the DBF? I bet Christof knows all the details... 😊
--
rk

-----Original Message-----
From: ProfoxTech [mailto:profoxtech-***@leafe.com] On Behalf Of ***@mbsoftwaresolutions.com
Sent: Thursday, January 25, 2018 2:03 PM
To: ***@leafe.com
Subject: Does the CDX file travel across the network for each time a field that can use an index is queried/updated/deleted/added? (was Re: Indexes on VFP tables--when do they get updated)

Perhaps a better question at this point: Does the CDX file get pulled
down locally for every query? I would think for ADD/UPDATE/DELETEs, it
would get updated on the LAN. (In this scenario, the database is on a
network folder; not local.)


_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@MWHPR10MB1774.namprd10.prod.outlook.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers
Alan Bourke
2018-01-26 10:47:28 UTC
Permalink
No the CDX doesn't get pulled locally when you open it or when you amend records.

You can test this by opening a table on the LAN in VFP, then running Sysinternals Process Monitor and filtering to process name = VFP9.EXE and path ends with .CDX.

Then do some seeks and updates and you will see that it is just reading/writing chunks of the CDX on the LAN.

Obviously if you're not taking advantage of Rushmore optimisation it will be doing big, time consuming scans of the DBF instead.
--
Alan Bourke
alanpbourke (at) fastmail (dot) fm

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@webmail.messagingengine.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Stephen Russell
2018-01-25 19:04:40 UTC
Permalink
I see a potential problem when multiple users are doing updates on the same
table.
What constitutes the server file to be updated?
How do the localized cdx files know to update when the server version is
updated?
Post by m***@mbsoftwaresolutions.com
Post by Fernando D. Bozzo
When the index is evaluated initially, VFP knows exactly what fields affect
what index, so really just affected indexes are updated.
CREATE TABLE test (field1 C(10), field2 I)
INDEX on field1 TO test_f1 additive
INDEX on field2 TO test_f2 additive
Add some data, wait a minute, replace one of the fields and do a FLUSH
FORCE, and you will see that only the affected index changes his timestamp.
I don't think that's a fair test to use IDX files since this example uses
Post by m***@mbsoftwaresolutions.com
a CDX file, and it's all in the same file.
I think it *is* a fair test for exactly that reason. If you see only
one of the IDX index files change, that suggests that VFP only updates an
index where it would make a difference.
I grant that it could be that that optimisation does not exist in the
CDX handling, but how likely do you think that is?
Sincerely,
Gene Wirchenko
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAJidMY+bKhtfZuKEzcG8WLW5sx+ku8D=+85r3O2E7GUW-***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Richard Kaye
2018-01-25 19:12:19 UTC
Permalink
I think VFP figured out how to handle multi-user concurrent updates a long time ago... At least until MS broke SMB... Multiple times... 😊
--
rk

-----Original Message-----
From: ProfoxTech [mailto:profoxtech-***@leafe.com] On Behalf Of Stephen Russell
Sent: Thursday, January 25, 2018 2:05 PM
To: ***@leafe.com
Subject: Re: Indexes on VFP tables--when do they get updated

I see a potential problem when multiple users are doing updates on the same
table.
What constitutes the server file to be updated?
How do the localized cdx files know to update when the server version is
updated?
Post by m***@mbsoftwaresolutions.com
Post by Fernando D. Bozzo
When the index is evaluated initially, VFP knows exactly what fields affect
what index, so really just affected indexes are updated.
CREATE TABLE test (field1 C(10), field2 I)
INDEX on field1 TO test_f1 additive
INDEX on field2 TO test_f2 additive
Add some data, wait a minute, replace one of the fields and do a FLUSH
FORCE, and you will see that only the affected index changes his timestamp.
I don't think that's a fair test to use IDX files since this example uses
Post by m***@mbsoftwaresolutions.com
a CDX file, and it's all in the same file.
I think it *is* a fair test for exactly that reason. If you see only
one of the IDX index files change, that suggests that VFP only updates an
index where it would make a difference.
I grant that it could be that that optimisation does not exist in the
CDX handling, but how likely do you think that is?
Sincerely,
Gene Wirchenko
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAJidMY+bKhtfZuKEzcG8WLW5sx+ku8D=+85r3O2E7GUW-***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

Report [OT] Abuse: http://leafe.com/reportAbuse/CAJidMY+bKhtfZuKEzcG8WLW5sx+ku8D=+85r3O2E7GUW-***@mail.gmail.com

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@MWHPR10MB1774.namprd10.prod.outlook.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who
Fernando D. Bozzo
2018-01-25 19:19:29 UTC
Permalink
Yeah, and that's why DBF files can be easily corrupted: You have many
clients trying to update the same files remotely over a network. Any
network microcut, any client virus or any client that collapse or reset is
a potential contributor to write bad info, resulting in the well known file
corruption, being some of the worst those that passes unseen (corrupted
data that does not corrupt the file structure) but that soon or later you
will find.

This problem is more noticed starting from Windows greater than 2003/XP,
for wich the SMB protocol have changed, privileging fast througputs and
optimizations for HTTP protocols at the expense of the now deprecated ISAM
files access.
Post by Stephen Russell
I see a potential problem when multiple users are doing updates on the same
table.
What constitutes the server file to be updated?
How do the localized cdx files know to update when the server version is
updated?
Post by m***@mbsoftwaresolutions.com
Post by Fernando D. Bozzo
When the index is evaluated initially, VFP knows exactly what fields affect
what index, so really just affected indexes are updated.
CREATE TABLE test (field1 C(10), field2 I)
INDEX on field1 TO test_f1 additive
INDEX on field2 TO test_f2 additive
Add some data, wait a minute, replace one of the fields and do a FLUSH
FORCE, and you will see that only the affected index changes his timestamp.
I don't think that's a fair test to use IDX files since this example uses
Post by m***@mbsoftwaresolutions.com
a CDX file, and it's all in the same file.
I think it *is* a fair test for exactly that reason. If you see
only
one of the IDX index files change, that suggests that VFP only updates an
index where it would make a difference.
I grant that it could be that that optimisation does not exist in
the
CDX handling, but how likely do you think that is?
Sincerely,
Gene Wirchenko
[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAGQ_Ju=_FxRdwzcJc6tp3yH3V0NOLXsq=H=FW=Qvx1_PGeG=***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Gene Wirchenko
2018-01-24 23:59:28 UTC
Permalink
At 12:21 2018-01-24, Ted Roche <***@gmail.com> wrote:

[snip]
Post by Ted Roche
<OldWarStory>
I once worked on an app where the original developer thought it would
be a "clever" idea to define index expressions as UDFs. Yes, it's
It seems to me that it might be useful in some very limited
circumstances.
Post by Ted Roche
possible. In his UDFs, he would switch work areas, open tables if not
already opened, look up values, and then return the value, cleaning up
work areas and tables as he went. For every index definition for every
record. A reindex with more than a couple hundred records brought the
entire system to its knees.
</OldWarStory>
That does seem like a lot of unnecessary work. Wouldn't the
tables needed already have been open? Or did he have SET
ARCHITECTUREASTRONAUT ON?

[snip]

Sincerely,

Gene Wirchenko


_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mtlp000084
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Loading...