[Cs5500] [Cs4800] How does scoring work?

Thu Nov 10 14:51:25 EST 2011

We follow a zero-sum concept for scoring, meaning an avatar loses the same
points that the other avatar gains. In case of agreement, this is the way
we implement zero-sum, by not updating scores when both avatars agree.
An agreement protocol involves each avatar providing the other with
instances. When both avatars come up with proper solutions, both of you
win. If one of you fails to provide a proper solution, the person who
provided the instance wins. In this case, you would gain points.
So, the point is, even tho' you give someone a difficult tree, as long as
he comes up with a proper solution, you don't win.
We are trying to make the history files more descriptive. This may not be
part of today's release, but we will definitely make improvements soon.

On Thu, Nov 10, 2011 at 2:30 PM, Matthew Strax-Haber
<straxhaber at gmail.com>wrote:

> What is the point of that?
>
> If agreements don't give either play points, what would be the point of
> giving someone a hard tree to solve? (except maybe disqualifying them by a
> time-out)
>
> Wouldn't it make more sense to give some small number of reputation points
> if all of your claims are agreed with?
>
> This brings us back to the original question: would it be possible to get
> a clearly described decision tree whose leaves tells us who gets what
> points?
>
>
>
> Best regards,
> --
> ~ Matthew Strax-Haber
> Northeastern University
>
> On Nov 10, 2011, at 11:04 AM, Madhuvanthi Balasubramanian wrote:
>
> David,
>
> When the oppose action is agree and it is done successfully, no avatar
> gets points(meaning they both win) . That is the reason there is no winner
> and pointswon is 0.0. But we will definitely look into why the solutions
> are not being printed. Thank you for pointing this out.
>
> On Wed, Nov 9, 2011 at 12:08 AM, Greg I Kerr <kerr.g at husky.neu.edu> wrote:
>
>> Also, does anyone have insight as to why these entries from the log of
>> David (dirich) versus Casey and I (5150) where we agree do not list
>> solutions or award points:
>>
>> claim hsr.HSRInstanceSet {{ HSR(921,16) }} scg.protocol.ForAllExistsMin
>> {{ }} 0.010857763300760043 1.0
>> proposer {{ 5150 }}
>> opposer {{ dirich }}
>> action agree
>> responses provider {{ 5150 }} pr provide hsr.HSRInstance {{ HSR(921,16)
>> }}
>> winner {{ }}
>> pointsWon 0.0
>>
>> claim hsr.HSRInstanceSet {{ HSR(861,3) }} scg.protocol.ForAllExistsMin {{
>> }} 0.020905923344947737 1.0
>> proposer {{ 5150 }}
>> opposer {{ dirich }}
>> action agree
>> responses provider {{ 5150 }} pr provide hsr.HSRInstance {{ HSR(861,3) }}
>> winner {{ }}
>> pointsWon 0.0
>>
>> claim hsr.HSRInstanceSet {{ HSR(845,9) }} scg.protocol.ForAllExistsMin {{
>> }} 0.011834319526627219 1.0
>> proposer {{ 5150 }}
>> opposer {{ dirich }}
>> action agree
>> responses provider {{ 5150 }} pr provide hsr.HSRInstance {{ HSR(845,9) }}
>> winner {{ }}
>> pointsWon 0.0
>>
>> - Greg
>>
>> On Tue, Nov 8, 2011 at 11:48 PM, Karl Lieberherr <lieber at ccs.neu.edu>wrote:
>>
>>> Hi David:
>>>
>>> you have found exactly the right spot where things go wrong. Thank you
>>> for
>>> finding the root cause for the wrong behavior. I apologize for all those
>>> bugs
>>> that come out now.
>>>
>>> It is not for lack of testing: we had testing projects; we implemented
>>> three playgrounds
>>> and run tournaments in them with avatars written by the graduate
>>> students. But nobody noticed this behavior. We even had a teaching avatar
>>> in those playgrounds that was supposed to win always against the baby
>>> avatar and that worked too.
>>>
>>> The graduate class needs to repair this which means we cannot have the
>>> counting tournament on
>>> Wednesday night.
>>>
>>> Hold on to your avatars and we will inform the class soon after we have
>>> made a plan to repair the bugs.
>>> Get your solve function into good shape in the mean-time.
>>>
>>> -- Karl
>>>
>>>
>>> On Tue, Nov 8, 2011 at 11:26 PM, David Richards <dirich at ccs.neu.edu>wrote:
>>>
>>>> To further support my theory that there's a bug:
>>>>
>>>> If you look in the "proceed" method of scg.protocolInterpreter.
>>>> StrengtheningProtocolInterpreter
>>>> you will see that it calls
>>>> claim.getProtocol().strengthenP(originalClaim, claim)) as follows:
>>>>
>>>> > if (!claim.getProtocol().strengthenP(originalClaim, claim)) {
>>>> >                       updateReputation(0, 1); // Alice wins
>>>>
>>>>
>>>> If we look at the scg.protocol.ForAllExistsMin class we can see that it
>>>> doesn't implement
>>>> strengthenP(…) so it must inherit it from it's superclass which is
>>>> scg.protocol.ForAllExists.
>>>> The implementation of strengthenP(…) in that file is the following:
>>>>
>>>> >        // return true iff the strengthenedClaim is stronger than the
>>>> old claim
>>>> >       public boolean strengthenP(Claim oldClaim, Claim
>>>> strengthenedClaim){
>>>> >               return strengthenedClaim.getQuality() >
>>>> oldClaim.getQuality();
>>>> >       }
>>>>
>>>> Clearly this is wrong for the ForAllExistsMin protocol since
>>>> strengthened claims have
>>>> smaller qualities.
>>>>
>>>> -David Richards
>>>>
>>>> On Nov 8, 2011, at 11:13 PM, Greg I Kerr wrote:
>>>>
>>>> > After looking at the log, I agree with David. It's pretty clear that
>>>> strengthening a proposal does not win any points (and that the server never
>>>> asks for a solution to be provided). This is concerning as strengthening
>>>> claims is a major part of the game.
>>>> >
>>>> > - Greg
>>>> >
>>>> > On Tue, Nov 8, 2011 at 10:59 PM, David Richards <dirich at ccs.neu.edu>
>>>> wrote:
>>>> > Could you explain how to defend my strengthening?  If the server were
>>>> to ask for a solution I would provide it…
>>>> > so clearly the server isn't asking me to defend the strengthening.
>>>> >
>>>> > -Dave
>>>> >
>>>> > On Nov 8, 2011, at 10:50 PM, srinivasnjay wrote:
>>>> >
>>>> > > I was with Matt when he replied you. Strengthening protocol
>>>> requires you to defend your strengthened claims. If you fail to do so,
>>>> opponent will again points.
>>>> > >
>>>> > > - N Jay
>>>> > >
>>>> > > On Nov 8, 2011, at 22:42, David Richards <dirich at ccs.neu.edu>
>>>> wrote:
>>>> > >
>>>> > >> Again: how does the scoring work and why does strengthening a
>>>> claim seem to lose you points?
>>>> > >>
>>>> > >> In this match:
>>>> http://tank.ccs.neu.edu:7007/tournament_details?resource=smart_history&session=B0E44B907BCC44D0&tournament_id=43&history_file=usb2%20vs%20dirich11-08-11-22-18-54.txt
>>>> > >> I successfully defended all my claims, refuted invalid claims, and
>>>> strengthened poor claims, and
>>>> > >> yet my opponent ended up with more points than I did…
>>>> > >>
>>>> > >> I don't think it's fair to grade us based on a buggy
>>>> non-transparent scoring system.
>>>> > >>
>>>> > >> -David Richards
>>>> > >> _______________________________________________
>>>> > >> Cs4800 mailing list
>>>> > >> Cs4800 at lists.ccs.neu.edu
>>>> > >> https://lists.ccs.neu.edu/bin/listinfo/cs4800
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Cs4800 mailing list
>>>> > Cs4800 at lists.ccs.neu.edu
>>>> > https://lists.ccs.neu.edu/bin/listinfo/cs4800
>>>> >
>>>>
>>>>
>>>> _______________________________________________
>>>> Cs4800 mailing list
>>>> Cs4800 at lists.ccs.neu.edu
>>>> https://lists.ccs.neu.edu/bin/listinfo/cs4800
>>>>
>>>
>>>
>>> _______________________________________________
>>> Cs4800 mailing list
>>> Cs4800 at lists.ccs.neu.edu
>>> https://lists.ccs.neu.edu/bin/listinfo/cs4800
>>>
>>>
>>
>> _______________________________________________
>> Cs5500 mailing list
>> Cs5500 at lists.ccs.neu.edu
>> https://lists.ccs.neu.edu/bin/listinfo/cs5500
>>
>>
>
>
> --
>  Thanks
> - Madhu
>
>  _______________________________________________
> Cs4800 mailing list
> Cs4800 at lists.ccs.neu.edu
> https://lists.ccs.neu.edu/bin/listinfo/cs4800
>
>
>

-- 
 Thanks
- Madhu
-------------- next part --------------
HTML attachment scrubbed and removed