Saturday, July 02, 2016

100 days CT streak

Today I reached a 100 day streak at Chess Tempo. Time to evaluate. When I started, my average rating was about 1650, now it is about 1750. When I quitted chess before my two year break, my end rating was somewhere between 1650 and 1700. I guess I would have reached this new average of 1750 anyhow, regardless of how I exactly train at CT. I think that it save to say, that any improvement from now on will be the direct result of my training efforts. The stage of improvement due to adaptation to the exercises is over. Any improvement from now on will be against the odds of adult chess improvement. When I reach an average of 2000, I consider my method a success, and I will be going to play chess again.

Let me be clear beforehand: I obviously have not yet found a way to improve consistently. Yet I am optimistic. It feels quite different than after a few months of salt mining. I narrowed down the goal I should head for rigorously. I would describe it as:

To improve the pattern recognition of the 18 most important tactical themes by building cues that enhance retrieval.

I tried a whole lot of different methods. None of them worked. The following idea emerged, by method of exclusion of everything else. It should start with the main idea of the combination. Before everything else, that idea must be recognized. It is always one of the 18 tactical themes that forms the basis (the clue) of the combination. All other tactical themes are just part of the preparation. They maintain the initiative. Once the main idea is found, the moves that lead to implementation of the idea, are usually found fast.

I can recognize the main tactical theme often flawless almost all the time. It only takes usually an awful lot of time. What is necessary, is to speed up the recognition of those 18 most important tactical themes. Therefore, I need to build a series of frameworks of cues.

I have gathered a database of about 100 problem position which I have failed to solve correctly within a reasonable time. I use FreeMind to toss around those problems, to categorize them from different angles of view. From an initiative viewpoint, I found the following categories (amount of puzzles between the square brackets [#] ).

Problem categories from the viewpoint of CCT
Every problem category needs its own approach. Every category needs its own framework of cues. I will start of course with the most frequent occurring failure categories. You can see that I often miss a double attack, especially when one target is not a piece, I frequently miss removal of the defender, and I often fail to find the right defense after capturing an incorrect sacrifice. These 3 categories alone are responsible for 54% of my failures.

It is no sinecure to build a framework of cues that is usable in many positions, since the combinations are so diverse. But I belief it is possible, though. I'm going to give it a try.
Building a framework


  1. There is the problem that you measure your performance with the problemset you use to do your training.
    Ct serve you problems "in your range of difficulty" this is usually a set of ~~ 10 000 problems ( 90++ (?) % of the problems you will see are in this "set" ) . In your settings you did "Avoid Rated Problems With Alternative wins" which does reduce the number of problems to somthing close to 7000. Your change in rating did increase the size of the set a little though. With 9532 blitz problems done you are now repeating many of the problems you get. Even if you dont have the impression that you have seen any of these problems before, the performance on duplicates is not the same as the performance on non-duplicates. There is one tactician at chesstempo who dramatically benefits from duplicates , thats a tactician with an outstanding chess memory though, he hardly fails at any problem twice.

    CT did compensate this effect of duplicates by a "duplicate reward reduction" in the past mainly to prevent rating drifts at the server. But then the development of the ratings of the tactician where .. ehm .. not motivating... The ratingdrift of the problems is compensated by different methods now ;)

    If you want to >>measure<< your performance it would be good to select other problems and or a different servers to do that independend to your trainingsset.

    You could reduce the problem of duplicates at CT daramatically by:
    Accepting duplicates ( increases the size of "your set" by a factor ~~10/7 )
    Change periodically ( once a month? ) from "easy" to "hard" and back in your "preferences" ( increases the size of "your set" by a factor ~~2 )

    Well i know.. lot of work.. just want to inform you that things are not easy.. which you already know ;)

    I did download my "history" and recalculated a ct-like rating only on non duplicates with the problemratings of "today" to compensate all "sideeffects"

    1. I'm working different now. The salt mine style approach to CT doesn't work. I have a problem set of only 100 now, which I maintain in Freemind. Studying a single position for a few hours is not unheard of. I want to distil the knowledge that is in the position, and focus on the knowledge that is reusable for other positions. I use CT for experimenting with different areas of attention. Overtime, I expect my problem set to grow to 500-1000 at max. I expect to find all the tactical knowledge you will ever need within those 500 positions. It is the art to distil it and make it usable.

  2. I have already started solving the puzzles at the level 1000-1400. They are new to me and I had not been solving them before. It will be interesting to see what CONCLUSIONS can be made. I am especially curious about the mistakes and counterplay (counterattack). In addition I will try to categorize the most difficult ideas and positions (puzzles).

    Recenly I have solved about 60-80 chess puzzles at CTS. I could barely believe what happened. I have been making such mistakes that it SHOWS I do not understand the concepts behind the positions. After I finish the puzzles from the (papier) workbook - I will give a try CTS puzzles. I hope some progress will be visible.

    My friend Tempo. Let me know what elements and hypotheses you are interested in while solving (easy) puzzles by me. If it is possible, I will include your ideas into the testing.

    PS. Your passion and curiousity is contagenous! I want to do a small part of work you have been doing for a year (when you came back to chess after long hiatus).

    1. So finally you have seen the light "Recenly I have solved about 60-80 chess puzzles at CTS. I could barely believe what happened. I have been making such mistakes that it SHOWS I do not understand the concepts behind the positions."

      We already talked about captures, what kind of different motives for captures are there (with examples)? As you can see in the picture in this post, I found already 6 different motives (be aware that our native English friends see motifs and motives as something different).

      It is difficult to predict what we are gonna find, hence it is difficult to say how you can help. Observations of any kind are always helpful. As you can read in my latest post, right now I'm interested in a way to prune the tree of analysis by looking at a few figures. Which moves accomplish the most. Any observation about the relation amount of attacks/tempi and the best move is welcome.

  3. A digression (or, down where I live, referred to as "chasing a rabbit through the briar patch"). . . please ignore if you are math-phobic!

    Let's assume that the set of "useful" tactical themes is composed of 18 members.

    (I think this is a lower bound, because there are qualitative differences between how individual pieces are used in these themes. For example, although we classify "forks" into a particular theme, the pattern recognizers for each type of piece are qualitatively different. That's another, different "rabbit.")

    If we assume that we are only going to "see" 2 of those themes in a specific sequence, then there are potentially P(18,2) different permutations or 306 different ways to use one theme followed by another (different) theme. If we extrapolate to 3 themes, P(18,3) = 4896. If we extrapolate to 4 themes (which I think is getting pretty far out on the likelihood scale), P(18,4) = 73440. (I used MS Excel's PERMUT function for the calculations; I don't trust my internal mathematical factorial function.)

    Why speculate in this way? Because I think that the pattern recognition "clues" that need to be "seen" change when there is a different ordering of the tactical themes. For example, if there is a discovered check FIRST, followed by a Knight fork of two major pieces SECOND, that is qualitatively different (requiring different "clues") from a Knight fork of two major pieces FIRST, followed by a discovered check SECOND. The order of utilization of the themes changes the perceptual pattern(s) that must be triggered in order to "see" the solution quickly.

    If there is any validity to that line of reasoning, then it becomes more imperative to find typical examples that can be "generalized" conceptually in order to provide the skills needed to recognize the appropriate "clues" in ANY position. I speculate that the required set of "triggers" will be different for each person, and that each person will have to investigate and determine for himself what is relevant. I don't think there is a universally applicable set of problems that will "teach" the needed skills. I've often been wrong before; this wouldn't be the first time I've followed a dying "rabbit" through the conceptual "briars."

    It will be fascinating to "see" your final problem set at the end of your investigation.

    Back to regularly scheduled programming of "vision". . .FORWARD! BACKWARD! FORWARD! . . .

    1. It has been long ago, that permutation stuff. But 18 themes COMBINE with 18 themes in 18x18=324 ways. When you have a combination of 6 themes, you still see 324 ways a theme combines with the next theme that directly follows it. And only that is relevant. You consider a theme only in relation to the theme that follows it directly.

      But this might very well be a useless discussion, since we don't know beforehand what we will be going to discover.

      I expect more from the discovery of a few simple rules. Rules that a better player uses without thinking about it. For instance a few rules how to escape a check. When must the king counter attack, when must the king defend, when must future checks be prevented at all costs, that kind of stuff. There are probably a few clues that can help you to decide.

      The same for decisions like "should I take with the bishop or the knight first?". Or "which target must be captured first?"

      Decisions are awfully time consuming, and a few rules might make a whole lot of difference.

      The final dataset is not relevant. Any 500 failures will do. In those 500 puzzles there is a world to discover. Most knowledge will not be found in just 1 position, you need to study a series of problems within the same category. That's where Freemind comes in, an ideal tool for categorizing puzzles from any angle of view.

    2. I started to categorize the puzzles conform the 324 combinations of themes. I discovered a new breed: the multi themed moves. Take for instance

      r1bq1rk1/4b1pp/4p3/pp1nNp2/3P4/1B6/PP1B1PPP/R2QR1K1 w - b6 0 1

      The first move has the themes double attack, clearance and pin. The second move for white has the themes pin, distraction and replace target in it.

    3. I concur with your idea that a few "rules" (heuristics?) can make a whole lot of difference in what we "see." I can glance at virtually any K+P vs. K position and map out the entire line of play until the Pawn is promoted (or stopped from being promoted). It is NOT because I can "see" every move that will be needed (I cannot "see" that far ahead), but because I have the requisite final goal patterns burned into memory (LTM). It's sort of a method of "stepping stones" (Tisdall) that alleviates the requirement to calculate. I think this is what needs to be done with all forms of tactics. Unfortunately, I know I stopped at the "familiarity" level for most of my tactical knowledge, which is insufficient for the level of skill desired. I'm working on that now. . .

      As you just discovered, things are considerably more complicated than just finding the "leitmotif" of a particular position. That (ultimately) was the point of my mathematical digression above.

      The "math" doesn't matter. The point is still to "see" what is available (motifs, themes) in a particular position. IMHO, that MUST be at a higher level of abstraction (conceptualization) than the individual moves, if we want to "see" rapidly rather than calculate slowly.