Notrump-raising strategies and hand evaluation

[[ Part 1: introduction, terminology, and evaluation of various
           MW-based strategies at total points and MP
]]

Copyright 2006 by Alex Martelli (aleaxit@gmail.com)

This work is licensed under the Creative Commons Attribution 2.5
License (see http://creativecommons.org/licenses/by/2.5/).


The purpose of this article is to initiate a study about how to best
evaluate balanced (BAL) bridge hands (4333 shapes) for the purpose of
bidding the optimal level of Notrump (NT) contract.

Specifically, for any given algorithm for hand evaluation (such as
High-Card Points, HCP: the usual scale, A=4 K=3 Q=2 J=1, commonly
attributed to Milton-Work, and thus also known as MW, or, when
disambiguation is needed, OMW for opener, RMW for responder), a
"responder raising strategy" (RSS) is specified by the RMW range
"low-high" (LH) such that responder will raise a 1NT opening bid to 2NT
with L<=RMW<H (it can be assumed that responder will pass with <L points
and raise to 3NT with >=H points). [[Note the use of an open interval,
to ease the specification of raising strategies that never raise to 2NT,
but rather always either pass or jump to 3NT: such RSS are simply and
naturally represented by a pair (L,H) with L==H]].

Similarly, an "opener acceptance strategy" (OAS) is specified by a
single threshold (T) such that opener will pass a 2NT raise when OMW<T,
and go to 3NT on such a raise when OMW>=T.  A "combined strategy" (CS)
is the pair (RSS, OAS) or equivalently the triple (L, H, T).  Not all
such triples give distinct results: in particular, when the RSS is an
empty interval (L==H), the choice of T does not influence the result. We
conventionally map all the CS in such equivalence classes to the CS
within the equivalence class with the lowest T, for definiteness.
Similarly, CS where opener always bids 3N on 2N are equivalent to those
in which responder bids 3N instead of 2N with the specified range, and
are similarly mapped to that same CS, not separately considered.  So, T
will always be opener's minimum when L==H, and will never be opener's
minimum when L<H.  Strategies where opener always passes 2N need not be
considered, as they are certainly dominated by those where responder
always passes 1N; so, T may be limited to opener's maximum (included).

In this study, we artificially restrict OMW and RMW to "interesting"
ranges, specifically 15-17 (a popular range for NT openers) and 8-10 (a
range where 3NT might or might not be optimal).

The distinct RSS considered in this study are therefore:
(8,8)   pass=never, 2N=never, 3N=8+
        CS: (8, 8, 15)
(8,9)   pass=never, 2N=8, 3N=9+
        CS: (8, 9, 16) and (8, 9, 17)
(8,10)  pass=never, 2N=8-9, 3N=10
        CS: (8, 10, 16) and (8, 10, 17)
(8,11)  pass=never, 2N=8-10, 3N=never
        CS: (8, 11, 16) and (8, 11, 17)

(9,9)   pass=8, 2N=never, 3N=9+
        CS: (9, 9, 15)
(9,10)  pass=8, 2N=9, 3N=10
        CS: (9, 10, 16) and (9, 10, 17)
(9,11)  pass=8, 2N=9-10, 3N=never
        CS: (9, 11, 16) and (9, 11, 17)

(10,10) pass=8-9, 2N=never, 3N=10
        CS: (10, 10, 15)
(10,11) pass=8-9, 2N=10, 3N=never
        CS: (10, 11, 16) and (10, 11, 17)

(11,11) pass=8-10, 2N=never, 3N=never
        CS: (11, 11, 15)

We have therefore a total of 16 CS to consider. For ease of naming, we
can map each number to a letter with the trivial code A=1, B=2, ... and
give each strategy a 3-letters name, from HHO (8, 8, 15) to KKO (11, 11,
15) -- specifically, the 4 relevant numbers for responder, 8 to 11, map
to the letters H, I, J, K, and the relevant numbers for opener, 15 to
17, map to the letters O, P, Q. Strategy names are therefore:
HHO HIP HIQ HJP HJQ HKP HKQ IIO IJP IJQ IKP IKQ JJO JKP JKQ KKO

To compare two strategies, we can simply simulate a long duplicate
"match" between them, using double-dummy play.  There may be distinct
results for each of several forms of scoring: total-points (TP),
vulnerable (V) or non-vulnerable (NV); IMPs, V or NV; board-a-match
(BM), equivalent to matchpoints (MP), where vulnerability does not
matter for the purposes of this study.

The tools used to generate and evaluate deals for this study are:
- 'dealer', http://www.dombo.org/henk/dealer.html, for hand-generation
- Bo Haglund's 'DDS', http://web.telia.com/~u88910365/, for double
  dummy evaluation
- Python, http://www.python.org, for everything else (as usual;-).

I owe special thanks to Bo Haglund for making DDS's source code
available to me for this research, so that I was able to port it to
MacOSX, integrate it more smoothly with Python, and run the result on
all of my Macs.  Bo's code's performance is truly excellent, about 0.65
seconds to evaluate a deal on each processor core of a Macbook Pro (for
a total throughput of over 10,000 deals per hour using both cores) and
1.27 on an iBook G4 (so I was able to use my iBook and Powermac to
crunch through the evaluations of yet more deals).

The results of all this crunching can be summarized as follows:

400000 hands in 330887.26 (0.83)
15-08 (62063): 2:3 3:8 4:45 5:469 6:4348 7:19839 8:28163 9:8320 10:841 11:27
15-09 (59132): 3:4 4:10 5:101 6:1305 7:10397 8:28747 9:15595 10:2780 11:192 12:1
15-10 (53394): 3:1 4:2 5:26 6:350 7:3971 8:20724 9:21546 10:6072 11:690 12:12
16-08 (47988): 2:1 4:1 5:93 6:1059 7:8475 8:23673 9:12532 10:2021 11:133
16-09 (44193): 3:2 4:5 5:18 6:256 7:3349 8:17257 9:17536 10:5164 11:596 12:10
16-10 (39691): 4:1 5:3 6:59 7:1009 8:9806 9:18322 10:8831 11:1616 12:43 13:1
17-08 (34714): 3:1 4:1 5:9 6:169 7:2576 8:13840 9:13900 10:3770 11:438 12:10
17-09 (31802): 5:3 6:32 7:741 8:7642 9:15039 10:7059 11:1239 12:47
17-10 (27023): 5:1 6:8 7:205 8:3023 9:11324 10:9495 11:2794 12:173

This means: 400,000 deals were generated and DD-evaluated (at a weighted
average time cost of 0.83 seconds per deal across the various Macs I
used).  Of these, 62063 times opener had 15 in front of responder's 8;
out of those cases, North declaring NT took 2 tricks in 3 cases, 3
tricks in 8 cases, 4 tricks in 45 cases, ..., 11 tricks in 27 cases.

Before moving on to comparisons, we may simply evaluate each CS on its
own by computing its total-points expectation, per deal, both V and NV
(like all other evaluations in the following, this uses the normal
tournament scoring of bonuses -- 50 for a part-score, 300 for a NV game,
500 for a V game -- no doubling, etc, is assumed):

TP expectations for all strategies
   HHO HIP HIQ HJP HJQ HKP HKQ IIO IJP IJQ IKP IKQ JJO JKP JKQ KKO
NV 157 167 169 169 164 160 142 175 177 171 169 148 165 156 143 131
 V 216 228 227 226 209 206 163 231 229 213 209 167 198 179 152 130

This table shows that, by small margins, the best CS at total points,
NV, is IJP, barely edging out IIO; V, vice versa, IIO barely edges out
IJP (and many others -- HIP, HIQ, HJP -- are not far behind).

IIO is the strategy where responder passes with 8 and bids 3N with 9-10;
IJP is the strategy where responder passes with 8, raises to 2N with 9
(to 3N with 10), and opener passes 2N with 15 but goes to 3N over 2N
with 16-17.

The fact that (very marginally) IIO, the more aggressive of the two,
wins over the more aggressive IJP when V, but loses when NV, confirms
the traditional wisdom that game-bidding at TP should be more aggressive
when V than when NV.  However, to understand exactly what's going on
here, it's worth examining the TP results (V and NV) of these two
strategies, IIO and IJP, in more detail:

TP expectations for strategy IIO (9, 9, 15):
   15-08 15-09 15-10 16-08 16-09 16-10 17-08 17-09 17-10
NV 101.5  81.6 188.4 121.1 187.2 284.5 136.6 288.7 363.0
 V  97.1  98.8 266.6 119.8 264.6 414.5 136.3 421.3 532.6

TP expectations for strategy IJP (9, 10, 16):
   15-08 15-09 15-10 16-08 16-09 16-10 17-08 17-09 17-10
NV 101.5  95.7 188.4 121.1 187.2 284.5 136.6 288.7 363.0
 V  97.1  84.4 266.6 119.8 264.6 414.5 136.3 421.3 532.6

The differences, of course, come only when opener has exactly 15 points,
and responder exactly 9: in all other cases, the two CS lead to the same
final contract, while, in the specific case 15-09, IIO leads to playing
3NT while IJP leads to playing 2NT.  In this case, the conditional
expectations of IIO are 81.6 NV, 98.8 V; those of IJP are 95.7 NV, 84.4
V.  The overall differences in expectation, as we can read in the
previous table, are of course much smaller than the -14.1/+14.4 seen in
this specific case, since the previous table is "watered down" by
averaging in all the cases in which these two CS play the same contract,
while here we're focusing on the cases in which contracts differ.

Let's drill down yet further, doing direct comparisons based on the
relevant line of the results summary table, namely:
15-09 (59132): 3:4 4:10 5:101 6:1305 7:10397 8:28747 9:15595 10:2780 11:192 12:1
1

TP comparisons for IIO vs IJP:
At NV
15-09, bid tr 9 vs 8:
 (4+10+101+1305+10397=11817)*(-50) + 28747*(-170) + (15595+2780+192+1=18568)*(+250)
  --> -835840 (-14.1 on 59132 deals)
At  V
15-09, bid tr 9 vs 8:
 (4+10+101+1305+10397=11817)*(-100) + 28747*(-220) + (15595+2780+192+1=18568)*(+450)
  --> +849560 (+14.4 on 59132 deals)

So this is where the -14.1 and +14.4 differences in conditional
expectations come from.  In the NV cases, there are 11817 deals in which
not even 2N makes; there, IIO loses 50 points (one undertrick) to IJP.
In 28747 deals, NS make exactly 8 tricks, so IJP's gain for playing 3N
(vs IIO's 2N) is 170 -- 120 for the value of 2N, plus 50 for the
undertrick.  In 18568 cases, IIO gains 250 points -- the difference
between the 300-points bonus for a NV game, and the 50-points bonus for
a partscore.  The grand total is -835840 over 59312 deals, and division
reveals that the weighted average, i.e., the expectation, is -14.1.

Things are _almost_ identical when V... except for the sizes of the
gains and losses: IIO's losing cases lose 50 points more (because the
undertrick costs 100 instead of 50), but the winning cases gain 200
points more (because the game bonus is 500 instead of 300), which,
overall, is enough to alter the sign of the result.

We don't really need to "play" all the 16*15/2 "matches" at total
points: simply comparing the TP expectations of the various strategies
gives us exactly the results of each "match", in terms of TP won or lost
per hand in average.  Here's another table, presenting these results
with higher precision:

TP expectations for all strategies
        NV      V
HHO 157.83 216.91
HIP 167.15 228.24
HIQ 169.24 227.14
HJP 169.24 226.11
HJQ 164.55 209.20
HKP 160.89 206.74
HKQ 142.49 163.07
IIO 175.73 231.29
IJP 177.82 229.16
IJQ 171.04 213.34
IKP 169.47 209.79
IKQ 148.98 167.21
JJO 165.06 198.75
JKP 156.71 179.37
JKQ 143.00 152.62
KKO 131.32 130.15


We can well expect matchpoint (board-a-match) results to be very
different, since at MP the size of the swing does not matter -- only the
swing's *sign* does.  Thus, in particular, there is no reason to
separate the analysis for V and NV, since vulnerability does not affect
the sign of the swing, only its size.  On the other hand, only
head-to-head comparisons between strategies matter, so we do need a
full, 16x16 table (of course, we do expect the main diagonal, where a
strategy "vies" against itself, to show perfect equality, while cells of
the table occupying symmetric positions wrt the main diagonal will have
to be exactly complementary).  However, to get a general idea of each
CS's general worth, we can show how it fares against the redoubtable
pair of Ron Result and Mary Merchant, who always bid 3N when it's fates
to make and otherwise stop at 1N; it's of course impossible to _beat_
Result-Mechant on any deal, so the performance of each CS can be shown
by the fraction on deals (out of 1000) on which the CS manages to _tie_
the board.

#boards/1000 tied against RM, all strategies
   HHO HIP HIQ HJP HJQ HKP HKQ IIO IJP IJQ IKP IKQ JJO JKP JKQ KKO
   470 517 540 543 550 524 484 622 647 632 628 566 633 614 567 529

IJP appears best here -- again, IJP is the strategy where responder
passes with 8, raises to 2N with 9 (to 3N with 10), and opener passes 2N
with 15 but goes to 3N over 2N with 16-17.  We also see respectable
performances from JJO (responder passes with 8-9, jumps to 3N with 10),
IJQ (as IJP, but opener goes to 3N over 2N only with 17), IKP (responder
never jumps to 3N, but rather passes with 8 and raises to 2N with 9-10;
opener goes to 3N over 2N with 16-17), IIO (responder passes with 8,
jumps to 3N with 9-10), JKP (responder passes with 8-9, raises with 10;
opener goes to 3N over 2N with 16-17).  In general, therefore, the
well-performing strategies (for such a "constructive" activity as
bidding game) at MP are more prudent than at IMPs -- again, this is
consonant with traditional bidding theory.  However, the "middle of the
road" strategies IJP and IIO, which were optimal at total points,
perform quite fairly at MP, too (particularly IJP).

For head-to-head comparisons of strategies at MP, we will present the
results as fraction of deals (out of 300) in which the first strategy
wins, net of the deals on which it loses (a negative numbers thus shows
that the second strategy wins the overall BM match). [[the number 300 is
chosen somewhat arbitrarily to let the table format readably...!]]

MP matches between all strategies, scores in boards/400
     HHO HIP HIQ HJP HJQ HKP HKQ IIO IJP IJQ IKP IKQ JJO JKP JKQ KKO
 HHO  +0 -43 -62 -65 -81 -62 -60 -60 -82 -80 -79 -59 -65 -62 -44 -23
 HIP +43  +0 -18 -21 -38 -18 -17 -41 -63 -61 -60 -40 -46 -43 -25  -4
 HIQ +62 +18  +0  -3 -19  +0  +1 -32 -54 -52 -51 -31 -37 -34 -16  +4
 HJP +65 +21  +3  +0 -16  +3  +4 -19 -41 -39 -38 -18 -36 -32 -15  +5
 HJQ +81 +38 +19 +16  +0 +19 +21 -13 -35 -32 -31 -11 -33 -30 -12  +8
 HKP +62 +18  +0  -3 -19  +0  +1 -23 -45 -42 -41 -21 -39 -36 -18  -2
 HKQ +60 +17  -1  -4 -21  -1  +0 -34 -56 -54 -53 -32 -54 -51 -33 -18
 IIO +60 +41 +32 +19 +13 +23 +34  +0 -21 -19 -18  +1  -4  -1 +16 +37
 IJP +82 +63 +54 +41 +35 +45 +56 +21  +0  +2  +3 +23  +5  +8 +26 +47
 IJQ +80 +61 +52 +39 +32 +42 +54 +19  -2  +0  +0 +21  +0  +2 +20 +41
 IKP +79 +60 +51 +38 +31 +41 +53 +18  -3  +0  +0 +20  +2  +5 +23 +39
 IKQ +59 +40 +31 +18 +11 +21 +32  -1 -23 -21 -20  +0 -21 -18  +0 +14
 JJO +65 +46 +37 +36 +33 +39 +54  +4  -5  +0  -2 +21  +0  +3 +21 +41
 JKP +62 +43 +34 +32 +30 +36 +51  +1  -8  -2  -5 +18  -3  +0 +17 +34
 JKQ +44 +25 +16 +15 +12 +18 +33 -16 -26 -20 -23  +0 -21 -17  +0 +15
 KKO +23  +4  -4  -5  -8  +2 +18 -37 -47 -41 -39 -14 -41 -34 -15  +0

The salient result here is that IJP "wins" all matches, albeit with very
narrow margins from IJQ, IKP, JJO. IJQ ties with IKP and JJO, and IKP
edges out JJO by a narrow margin.  Essentially, this confirms the
set of well-performing strategies, their rough ranking, and the
closeness among the few best ones, that emerged from the one dimensional
table where we compared each strategy against the artificially perfect
scores of Result-Merchant.


In future installments...: comparisons at IMPs (V and NV); other ways to
evaluate high-card strength, and their comparison with MW.