Thursday, April 12, 2012

A systematic approach to the move order problem

The move order problem.
In the previous post and its comments we saw what the core problem is when calculating. We recognize all kinds of patterns, but we have trouble to find the right move order due to the vast amount of possibilities. It takes too much time to investigate all possibilities.

Brute force.
Munich proposes to solve this problem by brute force. This means that he still wants to investigate all posibilities by trial and error, but tries to speed up the process by learning all kinds of patterns by heart.

Search strategy.
What I propose here is to add intelligence. Can't we find an intelligent strategy here that outperforms the brute force method? I have tried the brute force method myself for years and walked into the law of diminishing returns. If we are able to prune branches we can speed up the process by not calculating some lines. In stead of running around like a headless chicken (which is my usual state when in trial and error mode).

Of course I can't present you a ready to roll method straight from the start. But if we join our efforts we might be able to crack the problem at hand.

Automation of the strategy.
Once we have found a strategy we will of course find new ways to automate it and we will discover patterns that are related to this strategy and that can be learned by heart. But later is early enough to worry about that.

3 types of tactics.
There are only 3 types of tactics. Methods to gain wood.
  • The duplo attack which is based on performing two attacks at the same time while the opponent can defend against only one.
  • The trap which is based on lack of space.
  • Promotion.
These 3 methods have in common that they all work with attackers and their matching targets. Lets see if we can work out an example.

White to move. You can find the solution here.

First step: identify the attackers and their targets.
I  use the following method: if an attacker was able to make two moves in a row, which pieces would he grab? If an attacker can't grab a piece within two moves I consider it to be irrelevant.




Ra8 - Bc8
Ra8 c8 Qd8
Ba3 - Nf8
Ba3 f8 Bg7
Ba3 e7 Dd8
Nd5 e7 Kg8
Nd5 e7, b6 Bc8
Nd5 f6 Kg8
Nd5 f6, c7 Re8
Nd5 b6, f6 Nd7
Nf3 e5 Nd7
Bg2 h3 Nd7

We don't know yet which of these attacker-target combinations play a crucial role, but the combinations that do must be part of this list. This means we have ruled out a whole bunch of pieces that aren't worth investigating. Hence we pruned a lot of branches already. We can prioritize this list. The combinations with the highest target value and the lowest attacker value are the most interesting.

Step 2: Identify the duplo attacks, traps and promotions.
We know that in order to gain wood the attacker-target combinations must be part of a duplo attack or a trap or a promotion. It is easy to see that there are no pawns ready to be promoted. There are two pieces about to be trapped: the black queen and Bc8. There are the following duplo attacks possible:
Pin Ra8 Bc8 Qd8.
Knightfork Nd5-e7-Kg8-Bc8
Knightfork Nd5-f6-Kg8-Re8
This leads to the following shortlist:




Ra8 - Bc8
Ra8 c8 Qd8
Ba3 e7 Qd8
Nd5 e7 Kg8
Nd5 e7, b6 Bc8
Nd5 f6 Kg8
Nd5 f6 Re8

Again we managed to prune a whole lot of branches.

Step 3: Identify problems on the route.
We start with the highest rated targets.
Knightfork f6.
Let's investigate the knightfork on f6. If you look at the defenders and attackers of f6, black has the upperhand. There are 3 defenders against 1 attacker. There is no way to annihilate those 3 attackers.
Knightfork e7.
There are 2 attackers and 2 defenders. If I manage to annihilate 1 I have the upperhand.The Queen is overworked since it has to keep an eye on both Bc8 and e7.
Pin Bc8
In order to take advantage of the pin I must attack the bisshop one more time extra. I can do this only with the white knight on d5. There are 2 routes: via b6 and e7.
Black defends b6 twice so I can't get the upperhand there. e7 has already been investigated.
Taking the bishop with Rxc8 forces the queen into the knightfork e7 and diminishes blacks control over e7.
Trap Qd8.
With Be7, the Queen is trapped. The problem is that I don't have the upperhand on e7 yet.

Again we managed to prune a few branches.

Step 4: Identify the candidate moves.
The candidate moves that are left are now:
  • Ne7
  • Be7
  • Rxc8
We can be sure that if there is a winning tactic that it must start with one of these three moves.
From this moment on I'm inclined to use brute force and find out which move is the strongest by trial and error.

By not starting to calculate as a headless chicken right away we managed to prune the tree back from the 41 possible legal moves to 3 serious candidates. Without any serious calculation yet. We have needed pattern recognition along the way, but the pruning itself happened by reasoning.
The reasoning itself is at this moment a slow process. The next step is to automate whatever is possible from this process of reasoning. I'm sure if I do 100 problems this way, my mind will automate parts of this process. The move order problem isn't quite solved. But we have pruned largely the possibilities to investigate.


  1. The list of wrong moves at ct is:

    Move Total Average
    Blitz Std
    1.Ne7 1054 1891 1815
    1.Rxc8 550 1834 1766
    1.Bxf8 40 1807 1698
    1.Bd6 8 1862 1799
    1.Bb4 4 2100 1790
    1.Nc7 4 1979 1724
    1.Rb8 2 - 1708
    1.Bh3 2 - 1801
    1.Nb6 1 - 1773
    1.Ne3 1 - 1685
    2.Rxc8 2 1950 1766

    So 1604 from 1660 who did it wrong and 1500+ who did it right had the same opinion about that problem, thats ~99 %.
    Your analysis has to be right :-)
    You just need to do it muuch quicker than 33 sec because you need calculationtime to see wich is right.

  2. Afterwards I realized I can combine step 1 and 2 into 1 step without loosing correctness or completeness.

  3. I try to recall what I was thinking when I solved this puzzle.

    1.Rxc8 QxRc8 2.Ne7+ - hm, not possible.
    The Re8 defends the square e7.

    A knight check Nf6+ could fork the Re8, but the square f6 is guarded by the Qd8 and the Bg7.
    Lets try to distract them:
    1.Bxf8 Bxf8 2.RxBc8 QxRc8 3.Nf6 wins the exchange, but I had to sac my exchange. Furthermore, what is if not 1...BxBf8 but 1...Rxf8 - hm, then the previous calculation is valid.
    So 1.Bxf8 RxBf8 2.RxBc8 QxRc8 3.Ne7 and I win the queen.

    going back to
    1.BxNf8 BxBf8 2.Nf6+ QxNf6 - no, that does not win anything.

    And how about interfering between Qd8 and f6 with 1.Be7 ... oh, if the Re8 takes, then 1.Be7 RxBe7 2.RxBc8 QxBc8 3.Ne7+ forking king and queen.

    But has black to take? Where can the black queen go? a5 is covered by my Ra8, and b6 and c7 are covered by my knight.
    Is that it? look again... yes, thats it - I move 1.Be7!

    This was about what I was thinking in uhm 15-20 seconds?

    Hard to believe? Well, how long did it take you to read this text? 30-45 seconds? Then it is possible to think it in 15 seconds. Brute force rules!

  4. wrong move order: me and over 531 people went for Nf5 (about 80% of those who failed the puzzle failed it with this move). Same idea, but wrong move order:

    I simply did not see that after 1.Nf5 Qd7 is possible.

    I also had a look at the right move 1.RxRe8+ but I miscalculated that after 1.RxRe8+ RxRe8 2.QxRe8+ NxQe8 3.RxNe8+ Qf8 4.RxQf8 KxRf8 white is a knight up.

    I had a counting problem here.

    I guess most people had a counting problem here. Strange.
    Look at the comments of this puzzle.
    jsobo comment on 7th sept 2010. If you read his comment with the words "I just calculated that after the exchange of queens the knight or queen can block the final check."

  5. @Munich,
    Hard to believe? Well, how long did it take you to read this text? 30-45 seconds? Then it is possible to think it in 15 seconds. Brute force rules!

    Do I have to assume you didn't notice that you used the system I propagate? Cool!

  6. But I never trained a systematic approach. This I can tell you for sure. If this is what you propagate, then I can tell you I learned it from learning a lot of puzzles in the CT Blitz range 1150-1475.

    During my thinking I am aware of seeing patterns I learned. It gives me more breadth/width in my search tree. The picture I have is more clear, I calculate more accurately. I think this is brute force.

    I am often finding the solution by "logical reasoning" like you call it.
    But I think this logical reasoning itself needs pattern recognition very much. I see for instance, that the piece xy is pinned or overloaded, and hence this guardian is not so reliable. Then I look for attraction/distraction exploiting this weakness. But before I can work on exploiting the weakness, I need to see the pattern first.

    The trouble is not that I can see a pattern in 3 seconds, while others need 10 seconds (these 7 seconds difference dont matter much). The trouble is, that at the ply 3 or 4 you dont look for 10 seconds anymore. The very 1st move it does not matter if you see the pattern in 3 or 10 seconds.
    But why did so many people overlook the Re8, which guards the square e7?!
    They did not look 2 seconds into it after 1.Rxc8 QxRc8.
    They jumped to fast to move 2.Ne7+ and "next" 3.NxQc8.
    This "next" was thought very sloppy.
    That is why they failed. They had a look at it for 3 seconds and did not see 2...RxNe7.
    A simple move. A veeery simple pattern.

    If people look for half an hour on this puzzle, they wont fail it. Because they double check Ne7+ again and again, and they will see then that e7 is guarded.
    But they failed to see it in a second.

    It is also a bit about "you see what you expect to see".

  7. example of not seeing patterns fast enough:

    Now we look at 2 wrong moves:
    2.RxNd3 and 2.Rd5

    2.RxNd3 was the move I moved, too. And failed.
    What did I miss? The simple skewer 2...Rd8+.
    Why did I miss it? Because after I found the solution move after maybe 20 seconds I though I found it all and played quickly. I did not look at the position longer than 2-3 seconds.

    My thinking was slopy.
    I thought: "1.Rd4 black goes with his rook anywhere and then I take his knight".
    Black did go anywhere (I expected that) and I took the knight very fast.

    The second most common error was:
    People how moved this move probably saw the skewer.
    But they thought they found a way:
    after 2...Kg6 3.RxNd3 Rd1+ 4.Kc7 RxRd3 5.Ne5+ the black king and rook is forked. White wins his rook back.

    But this "clever" thinking has a hole:
    2...Ke4 is also possible. So the 2.Rd5-people have overlooked this possibility.
    Again I may ask "why?" --> because they did not look long enough. Only 3 seconds. And they didnt see it.
    That is why it is so important to improve in speed. We need to see the skewer in very little time, otherwise we blunder.
    The move Rd5+ had an average of 1958 in rating! 513 went for this move.

    The systematic approach leaves you with a lot of possible stupid blunders like these simple moves. The reason is very often the same: People did not think long about it at the critical point.
    It is no coincidence, that wrong moves are most often the 2nd move - not the 1st one!

  8. To become a better tactician you have to learn

    more pattern

    recall the pattern quicker

    and learn the common refutation pattern to every pattern = know the pattern "sharper", "deeper"

    The refutation training is missing. I ased Richard years ago for a defence training, thats a training where the tactician has to find the only survive move.
    Such a survive training will do the job to "calculate" quicker

  9. The blue Coakley has a defensive problem on every page.

  10. Though I would not mind a defense training, it doesnt matter that there is no defense training besides the tag "defensive move" (which is also winning instead of just surviving) and tags like "avoid stalemate" and "avoid perpetual check".
    All that matters is to become quicker.* And this we hopefully achieve by adding intelligence and training to apply this added intelligence quicker (probably achieved by repetition).

    *well, maybe other stuff matters, too, but at least I feel I can improve in speed, and this is the factor that helps most.