Testing tells the tale (SYS 3054) -- "BETWEEN(a,b,c)" equivalent to "a BETWEEN b and c"

[m***@mbsoftwaresolutions.com]

VFP9SP2 WestWind WebConnection App on WinServer 2012

I was looking at a notoriously slow query to see about optimizing it.
Using SYS(3054,11,"cmemvar"), I was able to do so, but during my
testing, I found that using MyDate BETWEEN Arg1 and Arg1 was the same
optimization-wise as BETWEEN(MyDate,Arg1,Arg2). Somewhere in history, I
thought that using VFP's native BETWEEN(field,d1,d2) was NOT Rushmore
friendly. However, testing with SYS(3054) showed otherwise!

Just an interesting find this weekend for me whilst debugging.

Carry on!
--Mike

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

[Kevin J Cully]

I was working at a place that did ran batch processes at night using VFP. One of the managers came up to me and told me of a process that used to take 30 minutes now was taking 5 hours. Combined with the other processes, this meant we couldn't complete the processing overnight any longer and meet our deadlines.

I pulled the latest code into my test environment, and via logging I traced it down to one SELECT statement. It turns out one of our developers changed the code to read "WHERE B=A" where prior it was written "WHERE A=B". (Thanks source code control for allowing me to pinpoint when the change was made!) That small code change meant it no longer could take advantage of Rushmore and was basically doing a table scan. SYS(3054) is forever my friend. We were back to 30 minutes for that process and back on schedule without any major effort or rework.

I spoke to the developer who made the change and she didn't realize that a small change like that would affect performance in that way. It was logically the equivalent behavior. A great learning moment.

-----Original Message-----
From: ProFox [mailto:profox-***@leafe.com] On Behalf Of ***@mbsoftwaresolutions.com
Sent: Sunday, March 18, 2018 2:34 PM
To: ProFox <***@leafe.com>
Subject: Testing tells the tale (SYS 3054) -- "BETWEEN(a,b,c)" equivalent to "a BETWEEN b and c"

VFP9SP2 WestWind WebConnection App on WinServer 2012

I was looking at a notoriously slow query to see about optimizing it.
Using SYS(3054,11,"cmemvar"), I was able to do so, but during my testing, I found that using MyDate BETWEEN Arg1 and Arg1 was the same optimization-wise as BETWEEN(MyDate,Arg1,Arg2). Somewhere in history, I thought that using VFP's native BETWEEN(field,d1,d2) was NOT Rushmore friendly. However, testing with SYS(3054) showed otherwise!

Just an interesting find this weekend for me whilst debugging.

Carry on!
--Mike

[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@CY4PR09MB2134.namprd09.prod.outlook.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

[m***@mbsoftwaresolutions.com]

Post by Kevin J Cully
-----Original Message-----
Sent: Sunday, March 18, 2018 2:34 PM
Subject: Testing tells the tale (SYS 3054) -- "BETWEEN(a,b,c)"
equivalent to "a BETWEEN b and c"
VFP9SP2 WestWind WebConnection App on WinServer 2012
I was looking at a notoriously slow query to see about optimizing it.
Using SYS(3054,11,"cmemvar"), I was able to do so, but during my
testing, I found that using MyDate BETWEEN Arg1 and Arg1 was the same
optimization-wise as BETWEEN(MyDate,Arg1,Arg2). Somewhere in history,
I thought that using VFP's native BETWEEN(field,d1,d2) was NOT
Rushmore friendly. However, testing with SYS(3054) showed otherwise!
Just an interesting find this weekend for me whilst debugging.
Carry on!
--Mike

Also found that the ALLTRIMs in some SQLs were defeating optimization as
well. Just a tip for others here! I don't code that way but somebody
in the history of this app did.

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

[Bill Anderson]

Mike,

I believe ALLTRIM()s in SQL doesn't "defeat" optimization, it's just that
you can't have ragged (meaning, varying length) indexes in VFP.

A tag on ALLTRIM(field) is padded out to the full length of the field.

Bill Anderson


On Mon, Mar 19, 2018 at 12:55 PM, <

Post by Kevin J Cully
-----Original Message-----

Post by Kevin J Cully
Sent: Sunday, March 18, 2018 2:34 PM
Subject: Testing tells the tale (SYS 3054) -- "BETWEEN(a,b,c)"
equivalent to "a BETWEEN b and c"
VFP9SP2 WestWind WebConnection App on WinServer 2012
I was looking at a notoriously slow query to see about optimizing it.
Using SYS(3054,11,"cmemvar"), I was able to do so, but during my
testing, I found that using MyDate BETWEEN Arg1 and Arg1 was the same
optimization-wise as BETWEEN(MyDate,Arg1,Arg2). Somewhere in history,
I thought that using VFP's native BETWEEN(field,d1,d2) was NOT
Rushmore friendly. However, testing with SYS(3054) showed otherwise!
Just an interesting find this weekend for me whilst debugging.
Carry on!
--Mike

Also found that the ALLTRIMs in some SQLs were defeating optimization as
well. Just a tip for others here! I don't code that way but somebody in
the history of this app did.

[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CA+O1enHZyeGt1xHaA7MH_aBUAVfXs-***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

[m***@mbsoftwaresolutions.com]

Post by Bill Anderson
Mike,
I believe ALLTRIM()s in SQL doesn't "defeat" optimization, it's just that
you can't have ragged (meaning, varying length) indexes in VFP.
A tag on ALLTRIM(field) is padded out to the full length of the field.
Bill Anderson

Hi Bill!

But it really doesn't do anything for you, right? I mean, VFP works
best when it's used for it's fixed-width strengths, right? (Keep in
mind, I haven't chosen to use VFP backends since 2004!)

--Mike

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/***@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

[Fernando D. Bozzo]

Using ALLTRIM() on idexes is a bad idea, the full field is always better
and faster, because VFP does not have to preprocess the value for
searching, for adding a new value and for reindexing.

Post by Bill Anderson
Mike,
I believe ALLTRIM()s in SQL doesn't "defeat" optimization, it's just that
you can't have ragged (meaning, varying length) indexes in VFP.
A tag on ALLTRIM(field) is padded out to the full length of the field.
Bill Anderson
On Mon, Mar 19, 2018 at 12:55 PM, <

Post by Kevin J Cully
-----Original Message-----

Post by Kevin J Cully
Sent: Sunday, March 18, 2018 2:34 PM
Subject: Testing tells the tale (SYS 3054) -- "BETWEEN(a,b,c)"
equivalent to "a BETWEEN b and c"
VFP9SP2 WestWind WebConnection App on WinServer 2012
I was looking at a notoriously slow query to see about optimizing it.
Using SYS(3054,11,"cmemvar"), I was able to do so, but during my
testing, I found that using MyDate BETWEEN Arg1 and Arg1 was the same
optimization-wise as BETWEEN(MyDate,Arg1,Arg2). Somewhere in history,
I thought that using VFP's native BETWEEN(field,d1,d2) was NOT
Rushmore friendly. However, testing with SYS(3054) showed otherwise!
Just an interesting find this weekend for me whilst debugging.
Carry on!
--Mike

Also found that the ALLTRIMs in some SQLs were defeating optimization as
well. Just a tip for others here! I don't code that way but somebody in
the history of this app did.

[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAGQ_Juk36oB8Xes_pF5Ka3exCwLCHrsFF=ybqvi_Oi+***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages

[Stephen Russell]

I agree with your statement. Any function you toss into your index has to
be run row by row to evaluate the correct value. This is not optimizable
because the data is being tweaked. Using INTs as keys saves you from this
nightmare.

Post by Fernando D. Bozzo
Using ALLTRIM() on idexes is a bad idea, the full field is always better
and faster, because VFP does not have to preprocess the value for
searching, for adding a new value and for reindexing.

Post by Bill Anderson
Mike,
I believe ALLTRIM()s in SQL doesn't "defeat" optimization, it's just that
you can't have ragged (meaning, varying length) indexes in VFP.
A tag on ALLTRIM(field) is padded out to the full length of the field.
Bill Anderson
On Mon, Mar 19, 2018 at 12:55 PM, <

Post by Kevin J Cully
-----Original Message-----

Post by Kevin J Cully
Sent: Sunday, March 18, 2018 2:34 PM
Subject: Testing tells the tale (SYS 3054) -- "BETWEEN(a,b,c)"
equivalent to "a BETWEEN b and c"
VFP9SP2 WestWind WebConnection App on WinServer 2012
I was looking at a notoriously slow query to see about optimizing it.
Using SYS(3054,11,"cmemvar"), I was able to do so, but during my
testing, I found that using MyDate BETWEEN Arg1 and Arg1 was the same
optimization-wise as BETWEEN(MyDate,Arg1,Arg2). Somewhere in history,
I thought that using VFP's native BETWEEN(field,d1,d2) was NOT
Rushmore friendly. However, testing with SYS(3054) showed otherwise!
Just an interesting find this weekend for me whilst debugging.
Carry on!
--Mike

Also found that the ALLTRIMs in some SQLs were defeating optimization

as

Post by Bill Anderson

Post by Kevin J Cully
well. Just a tip for others here! I don't code that way but somebody

in

Post by Bill Anderson

Post by Kevin J Cully
the history of this app did.

[excessive quoting removed by server]

_______________________________________________
Post Messages to: ***@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CAJidMYK7X1B0v14QLf_=wOfy9cLigiXodCG_+***@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to se