[Cs5500] Community Auditor - Results

[Karl Lieberherr]

Hi Tuba:

I forward this to the class. In the last week we want to polish the system
so that the Community Auditor
is happy with the tournament results.

We make an action plan tonight.

-- Karl

On Mon, Dec 5, 2011 at 1:27 PM, <tkoc at ccs.neu.edu> wrote:

> Hi Prof. Karl,
>
> We tried to run HSR and MMG between deterministic baby avatars and
> deterministic teacher avatars. We changed new Random() to new Random(seed)
> to make them deterministic. For MMG, we don`t have a problem, but HSR
> throws exception(socket timeout exception) when we use new Random(seed). We
> couldn`t understand why this happened. When we change it to new Random() it
> works again.
>
> Regarding the issues that need to be changed:
>
> 1) When a player is kicked out, his reputation is 98 and the other player
> gets 102. The amount is not important. Kicked one should get less
> reputation. But sometimes, when someone kicked out at the beginning the
> smart history looks like:
>
> {{ root }}  100.0
>  {{ tuba }}  100.0
>
> and auditor notifies this issue as violation. I think this could be a
> printing issue because one of the tournaments I ran, an avatar kicked out
> at the beginning and output was like above but the score were right. The
> player who is not kicked get 3 points as score.
>
> 2) Almost in all smart history files, there are actions that has no
> responses. The responses part is empty. Auditor notifies this.
>
> 3) Another problem is that sometimes pointsWon is 1.000017846308765E-10. I
> think reputation has a small difference because of the precision and smart
> history checker notifies violation.
>
> Regards,
>
> Tuba
>
> ----- Original Message -----
> From: "Karl Lieberherr" <lieber at ccs.neu.edu>
> To: tkoc at ccs.neu.edu
> Cc: "nanda bhat" <n89_nanda at hotmail.com>, "karan ramprakash Bhat" <
> karan at ccs.neu.edu>, "Madhuvanthi Balasubramanian" <
> balasubramanian.m at husky.neu.edu>
> Sent: Saturday, December 3, 2011 10:21:46 PM GMT -05:00 US/Canada Eastern
> Subject: Re: Community Auditor - Results
>
> Hi Tuba:
>
>
> thank you for the results.
>
>
> Please can you prepare a list of issues that need to be changed based on
> your smart history analysis tool. We will then discuss this list with
> Ahmed on Monday.
>
>
> Also run your smart history checker on the output of a tournament between
> deterministic baby avatars and deterministic teacher avatars. Are the same
> problems showing up? If yes, we can use YOUR tournament setup for
> regression testing changes to SCG court.
>
>
> -- Karl
>
>
> On Fri, Dec 2, 2011 at 11:33 PM, < tkoc at ccs.neu.edu > wrote:
>
>
> Hi Prof. Karl,
>
> I ran auditor for yesterday`s hsr and mmg tournaments and I have attached
> the output files.
>
> I realized that when one of the avatars is baby, generally there is
> violation.
>
> There are some files which has only two lines when someone kicked out at
> the beginning. It looks like:
>
> {{ asm }} 98.0
> {{ babyavatar }} 102.0
>
> The above one is correct. When someone is kicked out, the kicked player`s
> reputation should be less.
> But in some history files the output is like below and our checker
> notifies violation.
>
> {{ player1 }} 100.0
> {{ player2 }} 100.0
>
> Another point that I realized, some actions don`t have responses (provide
> and solve) and checker notifies violation.
> I couldn`t have a chance to look in detail but there are several
> violations because of wrong reputation as well.
>
> In addition to parsing smart history file, we have also included parsing
> raw history file to check if there is kick out and config file to check
> validity of solution for an instance.
>
> I hope the output files are readable. We tried to make it user friendly.
> If there is a violation in a smart history file. The title looks like
> *** VIOLATIONS IN - babyavatar vs asm12-01-11-22-31-18.txt - *** (There
> are violations in "babyavatar vs asm12-01-11-22-31-18.txt")
>
> And also we are planning to write a document how to use community auditor
> and what rules we implemented. I have committed the code to SourceForge but
> it is not in beh file. We should move them to the beh file as you
> suggested. And I think there is no permission to display some java files in
> SourceForge. We made changes in scg (OpposeAction, Refuting, Strengthening,
> Agreement) and scg.history.
>
> Best Regards,
>
> Tuba
>
-------------- next part --------------
HTML attachment scrubbed and removed