Author Topic: Is an update planned to create a more sophisticated RepeatAdvisor?  (Read 8291 times)

0 Members and 1 Guest are viewing this topic.

Chadokoro_K

  • Newbie
  • *
  • Posts: 1
    • View Profile
Hi, I love Stackz for studying Japanese terms and kanji, but since discovering it and its simple Leitner approach I have become aware of more sophisticated versions of Spaced Repetition learning that better learn how we learn and streamline the process even further. I'm referring to software like SuperMemo, Anki, Mnemosyne, Reviewing the Kanji (these last three are either web-based only or have that as an option).

Instead of a simple pass/fail these programs allow the person to rank how easy or difficult something was to recall and produce. For example, a "fail" is a fail, but when we get a kanji correctly there is a big difference between something dredged up with great effort, something that takes medium effort and something that we know so well it is akin to breathing. The simplest of the above-mentioned programs uses "fail" "pass" and "easy (pass)" with some other built-in modifications to mix the cards around. The others seem to be based on one of the SuperMemo algorithms and have five or six different grades that one can assign to each card. So the review period length is dependent on which box (column) it is in AND how easily you can recall and produce the material.

Of course the latest version of SuperMemo apparently has problems with "over-learning" (putting hard things off for too long based on having a run of easy to acquire items. That is why the others use modifications of older SuperMemo algorithms.

I would love to see a more sophisticated algorithm implemented in Stackz (in addition to the current modes). I want to keep using Stackz but these more sophisticated SRL systems are tempting me away. I'm sure the same is true of other users and potential customers.

I would appreciate it if you would let us know what your plans (and possible time-line?) are regarding this.

Thank you.

Chris

  • Administrator
  • Sr. Member
  • *****
  • Posts: 429
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #1 on: June 07, 2007, 02:18:51 AM »

Currently we focus on the beta test of the functionality in the Stackz 07 beta version. Algorithms may be added at a later point in time if they are needed. We don't have any concrete schedule at this point in time.

Please note that the Stackz fundamental function is to give an accurate picture of the proficiency state of all cards at one glance, and the algorithms (mainly RepeatAdvisor, and the other ColorModes) are placed on top of this visualization, intended for making automatic use of the classification if no manual processing is wished. This makes Stackz different from other applications that use an algorithm as their fundamental function, without visualizing the state: The Stackz main function is independent from the algorithms. This is useful if a manual interaction is needed (e.g. real-world constraints like exam dates have higher priority than theoretic perfect learning). Also, we can add new algorithms quite easily if needed without interfering with existing functionality.

For example, a "fail" is a fail, but when we get a kanji correctly there is a big difference between something dredged up with great effort, something that takes medium effort and something that we know so well it is akin to breathing.

Concerning the extended fail/pass function: This corresponds *somehow* to the 5 levels in Stackz where the cards will eventually be accumulated. Not in one single classification, but after several classifications, the unknown cards will be on the left and the known cards on the right. Thus, in Stackz, the cards are truly characterized, and classified according to their nature, not according the next scheduled review. This gives you a very good starting point for any operation, e.g. "learn unknown cards" or "refresh known but old cards". These are fundamentally different tasks, with different levels of effort even if they are recommended for the same day.

So what happens in reality, is - as always with Stackz :-) - you know what you are doing:

  • If you learn cards from the left side, you will probably not be so strict to yourself, and promote it easily.
  • If you repeat the cards from the right side, you will probably be more strict to yourself.
  • If you refresh an old file, without any ambitions of perfection, you will be less strict that if you prepare for an exam.
  • If you use the RepeatAdvisor, you know why you are using it and what level of proficiency you aim at.


I would love to see a more sophisticated algorithm implemented in Stackz (in addition to the current modes). I want to keep using Stackz but these more sophisticated SRL systems are tempting me away. I'm sure the same is true of other users and potential customers.

Check those tools out, and see what is best for you. Preferences for tools differ in the same way the learning styles differ.
But what algorithm exactly would you wish?
Generally speaking, we believe that one should not try to find exaggerated precision in the whole process. The two difficulty estimations that can be used as basis for the RepeatAdvisor (promotion level, and success ratio) along with the assigned repetition delay are already very sophisticated. We are convined that what helps in reality is a simple tool that serves as a reminder for the constant learning actions, that will show you where your weak spots are, that will motivate you to continue improving your skills and somehow reward you for your efforts. All those things together bring value to the student, not the perfect moment for repetition alone.

Note:
The extended fail/pass function, translated to the Stackz world, would be a "directly send to level x". This would speed up the classification, but at the cost of lower accuracy (the success ratio is based on repeated tests). In addition, the user interface would become more complex, which we try very very hard to keep at an absolute minimum. But this has been requested before, and if more people wish it, we will introduce that one day. But not in the next release :-)


stackzrockz

  • Newbie
  • *
  • Posts: 25
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #2 on: February 07, 2008, 09:26:59 AM »
I think what Chadokoro_K is referring to is the difficulty of reviewing in Stackz.  I feel like Stackz is an excellent tool for cramming large amount of words into my head on a daily basis (10s to 100s of words).  However, for keeping large amounts of words in my brain, I feel like Stackz is at a bit of a disadvantage compared to programs that use spaced repetition.  It would be cool to have a spaced repition mode along with the other modes if it was well implemented.

Chris

  • Administrator
  • Sr. Member
  • *****
  • Posts: 429
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #3 on: February 08, 2008, 08:38:46 AM »
It is not advertised as such, but the RepeatAdvisor ColorMode is exactly what others now call spaced repetition!

Arqui3D.com

  • Newbie
  • *
  • Posts: 27
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #4 on: February 09, 2008, 11:40:37 PM »
In fact, RepeatAdvisor is the only mode I use. I dare to say that it is the method that more closely resembles how memory works. And you can fine-tune to your needs by going to File Options > Color modes > Advanced.

When I first started using Stackz!, I kept comparing it to more automatic systems based on a geometrically increasing space between repetitions, and I thought that 5 levels was not enough. At that time, I focused more on memorization than learning. And I made no progress for years, because I was so focused on reviewing my lists that I didn't have time to actually learn the language. And when my expired cards accumulated, I felt discouraged and often abandoned my Japanese studies for several months.

Now I have come to understand that you should concentrate on learning and practicing, rather than memorizing. So now I have come to appreciate the semi-automatic nature of Stackz!, and have calibrated it with less stringent options.
Windows XP Pro SP2
Core 2 Duo E6300
2 GB DDR2
nVidia GeForce 7300 GS with latest drivers

bkzk7188

  • Newbie
  • *
  • Posts: 18
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #5 on: May 16, 2008, 04:38:51 PM »
I'm a long-time Stackz/PocketStackz user, and spread out on 7 files I have a total of ~25000 entries which I review daily using RepeatAdvisor. Because of the sheer volume and approx. 85% of the entries having reached Level 5, I've tweaked the repeat times beyond "slow" and the current one for Level 5 is set to 200 days. Anything lower than that would have me constantly repeating words that I know far too well, and I don't want to spend more than 1 hour every day.

A possible solution to my (and other fellow long-time learners') problem would be a "RepeatAdvisor+" ColorMode. Simply increasing the number of levels could mess up the graphical representation, although thin columns wouldn't be a problem for me. An alternative solution would be if it turned Level 5 into some kind of "plus ultra" level where the repeat time would increase with every correct answer. This could be done using an algorithm OR simply by defining a longer series of repeat times.

One, very cumbersome, way around this would be splitting every file into two (or more), where the first one is set to "slow" and the second one would pick up where the first one ended, having longer repeat times. It would be a pain constantly moving entries between these lists, though.

I've been looking to other programs (Anki, Supermemo etc), but they really can't compare to Stackz so I'd love this feature to be added in a future version. Making it one of many ColorModes wouldn't disturb anyone, while putting Stackz further ahead of the competition by making it a viable alternative for long-time learners.
« Last Edit: June 17, 2008, 10:03:18 AM by bkzk7188 »

Chris

  • Administrator
  • Sr. Member
  • *****
  • Posts: 429
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #6 on: May 16, 2008, 06:01:35 PM »
An alternative solution would be if it turned Level 5 into some kind of "plus ultra" level where the repeat time increasing with every correct answer. This could be done using an algorithm OR simply by defining a longer series of repeat times.

Yes, I like that idea! It could be easily integrated in the current RepeatAdvisor: If the difficulty estimation is set to "promotion level" in the advanced dialog of the RepeatAdvisor options dialog, the refresh interval for level 5 could be extended with an additional option. As you suggest, adding a factor which multiplies the interval in the case of consecutive positive tests in level 5 would be quite simple.

Thank you for that suggestion, we'll think about adding this function to a future version.

Chris

  • Administrator
  • Sr. Member
  • *****
  • Posts: 429
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #7 on: July 06, 2008, 07:23:23 PM »
An improved version with this switch has been released today!

Please download the current release from the member zone.

napurenon

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #8 on: July 07, 2008, 08:38:34 PM »
I'm very glad to hear about this exciting new feature. Are you planning to add it to Pocket Stackz too?

Chris

  • Administrator
  • Sr. Member
  • *****
  • Posts: 429
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #9 on: July 21, 2008, 07:25:34 PM »
The latest version of PocketStackz includes the new features of Stackz 7.1 as well!

Please visit the members zone for the current build.
A direct link is here.

tarix

  • Newbie
  • *
  • Posts: 20
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #10 on: October 07, 2008, 07:55:52 AM »
The day this new feature was implemented I revived all of my StackZ! files and completely uninstalled Anki.  For me and my learning it represents a real upgrade from when I started using StackZ! 2.5 years ago.

However, there's a poorly implemented feature of Anki that StackZ! could implement very very well.  What I am about to describe is currently implemented on the website Remembering the Kanji.

In total there are 7 stacks.  There are the five stacks that we are all familiar with that give us an overview of how well we know any particular card.  However in RtK there is a stack for cards we have not yet studied or learned.  There is also another stack for cards that we used to know, but have recently failed.

Since programmatically these two new stacks are the same I'll just call it Stack 0.  Stack 0 only has two states.  Never learned and learned (but failed).  Regardless of their state all cards in Stack 0 move to Stack 1 when they are studied and answered "Yes, I know this".

From a user perspective I think the easiest way to visualize this to have seven stacks total.  The far left being cards you have never studied and learn and the one to the right of that being cards that used to be in the main StackZ! but have been failed and need to be restudied.

Thus, in principle, you would use the "Study/Learn" mode of StackZ! on the left two stacks and the "Test" mode on the right 5.

Alternatively, since changing the number of stacks might be a pain from the interface perspective it would still be nice to allow this sort of visualization.  That is the first stack is never studied cards, the second is failed cards, the next two are the same as the current second and third stacks, then the final stack is a combination of the last three stacks.  (This system only works well if you start doubling the time taken after stack 3.  Since research shows that's a good system anyway it's not a big limitation.)

Chris

  • Administrator
  • Sr. Member
  • *****
  • Posts: 429
    • View Profile
Re: Is an update planned to create a more sophisticated RepeatAdvisor?
« Reply #11 on: October 07, 2008, 10:40:59 PM »
Thank you for your continuous useful input, Tarix!

In fact it was only three days ago when I started implementing what I have mentioned in this thread more than one year ago:
Note:
The extended fail/pass function, translated to the Stackz world, would be a "directly send to level x". This would speed up the classification, but at the cost of lower accuracy (the success ratio is based on repeated tests). In addition, the user interface would become more complex, which we try very very hard to keep at an absolute minimum. But this has been requested before, and if more people wish it, we will introduce that one day. But not in the next release :-)

Now the time of this feature has finally come, and I think I found a way to integrate the new feature seamlessly in the existing context.

I added a new "promotion system" called "manual distribution" that painlessly extends the existing two promotion systems:


Now the bigger departure is to get rid of the arrow buttons in the study dialogs. If this system is selected, we need some sort of direct "send to" buttons.
Something like that:



Note that you can define your own labels for the five levels and use them for whatever purpose you want. Of course it makes sense to use this in combination with the RepeatAdvisor ColorMode, where you can define the repetition delay for each level individually:



I feel that there is little complexity added to the program; in fact many existing concepts can be easily extended with the new ideas.

Things that must be sorted out are:
  • In this mode, one would probably expect that the entries are refreshed even if moved to the left. Stackz up to now only considers a positive test as a relevant point in time; negative tests are not resetting the appearance delay of a card (it may come at a later day if moved to a column with a longer delay, but the wait period will only consist of the delay difference). Now I feel that this will have to change for the new mode - if the card is sent to a level on the left, it should NOT appear until the entire waiting time of that level is over - after all I sent it intentionally to that column with that delay in mind. That's why I tried adding the option "Mark entries as refreshed" on the bottom of the first screenshot. Currently it's hardcoded to the suboptimal version as the checked radio button indicates.
  • Now this "mark entries as refreshed" option could become a global option, allowing the user to adjust the refresh style for the other modes as well. I guess then it would be easier to understand for a new user because the overall feature would be more symmetric..?
  • Another side effect comes from the fact that the proposed design intentionally allows treating the 5 levels as unrelated decks of cards, not necessairly as linked steps in a process. I could e.g. name them according to the days of the week, and simply process the "due stack" each day, manually pushing the card to the day when I will review it again. Or I might want to separate the words according to their importance... or any other criterion, I'm sure the users will have other ideas. And the existing concept would allow that without problems as long as the user knows what he is doing. BUT: what happens to the "success count", "failure count", "success ratio" color modes? These statistics are built when moving the cards across the levels... The flexibility of allowing arbitrary level purposes introduces a problem here... one solution would be to add another option to indicte the type of dependency between the chosen level names, e.g. a checkbox "[ ] levels represent increasing proficiency" or something like that...

OK there were many thoughts in here, I hope it was not too confusing. It's mainly a "flexibility versus complexity" tradeoff.
It would be interesting to hear your comments! I will upload the current version in the latest version section for anyone interesteed.