Ideosphere Forum

fx-discuss: Re: fx-discuss: Re: fx-discuss: Fwd: FX Claim: Tran – Machine translation by 2015

Author: Neal Gafter
Conversation: fx-discuss: Fwd: FX Claim: Tran – Machine translation by 2015 ( prev | next ) reply!
Topic: fx-discuss ( prev | next )
In-Reply-To: Neal Gafter's post
Date: Sat Jan 27, 2018 02:15 pm
Neal Gafter
Neal Gafter




I am looking at judging the FX claim Tran
<http://www.ideosphere.com/fx-bin/Claim?claim=Tran. The market seems to
think that this claim has not been satisfied, and I have not found evidence
that it has been satisfied. Can any market participants present evidence
that they think should weight toward a TRUE judgment? Please respond within
the next 7 days if you have evidence to submit for consideration.

Cheers,
Neal "Loophole" Gafter

On Sat, Jan 20, 2018 at 1:39 PM, Neal Gafter <neal@gafter.com> wrote:

> I am still around. I will start looking at this in a few days.
>
> On Sat, Jan 20, 2018 at 9:18 AM chrisran.bma e-mail <
> chrisran.bma@virgin.net> wrote:
>
>> Hi,
>>
>> The due date for this claim 2017/12/31 has passed.
>>
>> Does anyone think or preferably have evidence to suggest this claim is
>> true or false?
>>
>> Is the judge, Neal Gafter aka loophole still around?
>>
>>
>>
>> I have previously written things like:
>>
>>
>> Findings of the 2017 Conference on Machine Translation (WMT17)
>>
>> http://www.statmt.org/wmt17/pdf/WMT17.pdf
>>
>> (September 7-8, 2017)
>>
>> and
>>
>> Findings of the 2016 Conference on Machine Translation (WMT16)
>>
>> http://www.aclweb.org/anthology/W16-2301
>>
>> (11-12 August 2016)
>>
>> Perhaps the following looks like what we want:
>>
>> [quote]
>>
>> 5.5.2 Human evaluation results
>>
>> Table 35 includes DA results for English-German and Table 36 shows
>> results for German-English APE systems. Clusters are identified by grouping
>> systems together according to which systems significantly outperform all
>> others in lower ranking clusters, according to Wilcoxon rank-sum test.
>>
>> # Ave % Ave z System
>>
>> ___________________________
>>
>> − 84.8 0.520 HUMAN POST EDIT
>>
>> ___________________________
>>
>> 1 78.2 0.261 AMU
>>
>> 77.9 0.261 FBK
>>
>> 76.8 0.221 DCU
>>
>> ___________________________
>>
>> 4 73.8 0.115 JXNU
>>
>> ___________________________
>>
>> 5 71.9 0.038 USAAR
>>
>> 71.1 0.014 CUNI
>>
>> 70.2 −0.020 LIG
>>
>> ___________________________
>>
>> − 68.6 −0.083 NO POST EDIT
>>
>> Table 35: EN-DE DA Human evaluation results showing average raw DA scores
>> (Ave %) and average standardized scores (Ave z), lines between systems
>> indicate clusters according to Wilcoxon rank-sum test at p-level p ≤ 0.05.
>>
>> [/quote]
>>
>> Seems to indicate that human translation is better than machine
>> translation, but of course that doesn't guarantee that there isn't a better
>> translation program somewhere from pre 31 Dec 2015 that simply didn't
>> attend the conference.
>>
>> Still if human level translation existed in 2015, you would not expect to
>> read things like
>>
>> [quote]
>>
>> This steady improvement has been mainly driven by the massive migration
>> to the neural approach, which in 2016 allowed the winning system to achieve
>> impressive results
>>
>> [/quote]
>>
>> I don't believe there is a program that can justifiably claim "equal or
>> better average quality, as professional human translations" but proving a
>> negative is difficult. I suggest if there was such a program it would be
>> big news, not difficult to find, and conference findings would be markedly
>> different to those linked above.
>>
>> Not sure how much more a judge might want before deciding how to judge
>> the claim. Are there any more authoritative events or other event before
>> claim deadline of 31 Dec 2017? (Note program has to exist by 31 Dec 2015
>> and translations have to 'be of comparable cost and turnaround time'.)
>>
>>
>> The comparable cost and turnaround time requirement seems to me to
>> indicate that secret research would not qualify.
>>
>>
>> Seems like an obvious false to me.
>>
>> Regards
>>
>> Chris Randles
>>
>> (crandles 7886)
>>
>> Disclosure I hold -3603 in this claim
>>
>>
>>
>>
>

source



All trademarks, copyrights, and messages on this page are owned by their respective owners.
Forum: Copyright (c) 2000-2001 Javien Inc All rights reserved. Distributed under the GPL