From owner-fx-discuss@ideosphere.com Sat Jan 27 14:15:48 2018
Received: from ideosphere.com (localhost [127.0.0.1])
	by ideosphere.com (8.13.8/8.13.8) with ESMTP id w0RJFm4L024464
	for <fx-discuss-outgoing@ideosphere.com>; Sat, 27 Jan 2018 14:15:48 -0500
Received: (from majordomo@localhost)
	by ideosphere.com (8.13.8/8.13.8/Submit) id w0RJFmpw024463
	for fx-discuss-outgoing; Sat, 27 Jan 2018 14:15:48 -0500
Received: from mail-vk0-f49.google.com (mail-vk0-f49.google.com [209.85.213.49])
	by ideosphere.com (8.13.8/8.13.8) with ESMTP id w0RJFlqM024460
	for <fx-discuss@ideosphere.com>; Sat, 27 Jan 2018 14:15:48 -0500
Received: by mail-vk0-f49.google.com with SMTP id n132so2239446vke.2
        for <fx-discuss@ideosphere.com>; Sat, 27 Jan 2018 11:15:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20161025;
        h=mime-version:sender:in-reply-to:references:from:date:message-id
         :subject:to;
        bh=GNx5v9HFdd+6RveNWdQghTGubhFxLhxcdUVw3YzQk34=;
        b=I3VfiM77XJB/oOMhjMnbx4KsVra/aofMz2O2Wj7FFzOI4OOVN+YQrEZYYN6OyXbn2j
         78iQXI6ehpzLDBD4DW91j9TIo36Jtb90neIhSnC1M4Ag8qsgNHZNS0HXhdN00bpqV7WI
         D4jyh929QaFjF7edTP+Yz0gMci0QbdULj1x64iuo47iccn3EVxrOZWJ4V3vCpDwBwMUl
         pY42wyHVVbXetD8/ZN1BULMEETEOpG8V9t/J1/F2imxJRE0PxrOe+Rp76nq2bqVyD/AS
         FzmxcvmTPIUjNf7reT2/T4j0XMf0dqaxhKcxKe4outvvW6zeM+GZZcKUanO5Y/+eSAt+
         fQEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:sender:in-reply-to:references:from
         :date:message-id:subject:to;
        bh=GNx5v9HFdd+6RveNWdQghTGubhFxLhxcdUVw3YzQk34=;
        b=VaXdh5FWGRVtAZY+WLbA/zDunLu7mFXqbAOV3jp3y/GWx4NIky77v8s27ABPdUWSAd
         PoNamxXfs9o9IkZMtMf96ADvGjB3URuELm55A1IpL9dTE2kWgH5kiiYiY4044SFjszuz
         miU7iGJj3/zMEr3BeHkTIqyaXKtm2YnkZKZ2ISBJh5sYp7l3pTCdRcHLBSh+5garhxM1
         J3GJ1WgsYxSVQ1jKmAOR+A/2V+YNSlqC8V0et16b7LVkCUYDjib3Xyrft32hx4S+6vch
         1mZbg6UImnGUIq8iEZ6bkIb1P7N9R5gFlQFP8TrTcPauovGI6bKLcxsRfyA8HcDplUW0
         cO3g==
X-Gm-Message-State: AKwxytdall73kFESDZTdeQ1/fiQuKwYP6Mcel2lVHJK9dagXt2wuNyz7
	frZjwH+UUv0lFfjcQ5G/6hc4ngFElAD/Anc+HjtVpg==
X-Google-Smtp-Source: AH8x226wG48xw1hVxAzEeRKkIOQTZ9HCaW4SNiFMzAgsPwE2TactJCwY7e0gJkotk8TNam2G2m9GPsAH/s/3X1KQrpw=
X-Received: by 10.31.254.201 with SMTP id l192mr13977156vki.153.1517080547066;
 Sat, 27 Jan 2018 11:15:47 -0800 (PST)
MIME-Version: 1.0
Received: by 10.176.19.239 with HTTP; Sat, 27 Jan 2018 11:15:06 -0800 (PST)
In-Reply-To: <CAD=nd8cYwgnVeOmjHDsRSbtO0_=iEG-K5nuC4wZfdnhL-2gNVA@mail.gmail.com>
References: <862447574.1932061.1515241909823.JavaMail.open-xchange@oxbe25.tb.ukmail.iss.as9143.net>
 <556361200.2395480.1516457867962.JavaMail.open-xchange@oxbe21.tb.ukmail.iss.as9143.net>
 <CAD=nd8cYwgnVeOmjHDsRSbtO0_=iEG-K5nuC4wZfdnhL-2gNVA@mail.gmail.com>
From: Neal Gafter <neal@gafter.com>
Date: Sat, 27 Jan 2018 11:15:06 -0800
X-Google-Sender-Auth: KSI7XC970L5n8Ebm5KxNu_u55rk
Message-ID: <CAD=nd8cgcpnMNT=C4Hcq0skAdfMcgovXkoTGBG6K58mCu=faaw@mail.gmail.com>
Subject: fx-discuss: =?UTF-8?Q?Re=3A_fx=2Ddiscuss=3A_Re=3A_fx=2Ddiscuss=3A_Fwd=3A_FX_Claim=3A_Tran_?=
	=?UTF-8?Q?=E2=80=93_Machine_translation_by_2015?=
To: fx-discuss <fx-discuss@ideosphere.com>
Content-Type: multipart/alternative; boundary="94eb2c14af701a89850563c6d782"
Sender: owner-fx-discuss@ideosphere.com
Precedence: bulk
Reply-To: fx-discuss@ideosphere.com

 
 I am looking at judging the FX claim Tran
<http://www.ideosphere.com/fx-bin/Claim?claim=Tran>. The market seems to
think that this claim has not been satisfied, and I have not found evidence
that it has been satisfied. Can any market participants present evidence
that they think should weight toward a TRUE judgment? Please respond within
the next 7 days if you have evidence to submit for consideration.

Cheers,
Neal "Loophole" Gafter

On Sat, Jan 20, 2018 at 1:39 PM, Neal Gafter <neal@gafter.com> wrote:

> I am still around. I will start looking at this in a few days.
>
> On Sat, Jan 20, 2018 at 9:18 AM chrisran.bma e-mail <
> chrisran.bma@virgin.net> wrote:
>
>> Hi,
>>
>> The due date for this claim 2017/12/31 has passed.
>>
>> Does anyone think or preferably have evidence to suggest this claim is
>> true or false?
>>
>> Is the judge, Neal Gafter aka loophole still around?
>>
>>
>>
>> I have previously written things like:
>>
>>
>> Findings of the 2017 Conference on Machine Translation (WMT17)
>>
>> http://www.statmt.org/wmt17/pdf/WMT17.pdf
>>
>> (September 7-8, 2017)
>>
>> and
>>
>> Findings of the 2016 Conference on Machine Translation (WMT16)
>>
>> http://www.aclweb.org/anthology/W16-2301
>>
>> (11-12 August 2016)
>>
>> Perhaps the following looks like what we want:
>>
>> [quote]
>>
>> 5.5.2 Human evaluation results
>>
>> Table 35 includes DA results for English-German and Table 36 shows
>> results for German-English APE systems. Clusters are identified by grouping
>> systems together according to which systems significantly outperform all
>> others in lower ranking clusters, according to Wilcoxon rank-sum test.
>>
>> # Ave % Ave z System
>>
>> ___________________________
>>
>> − 84.8 0.520 HUMAN POST EDIT
>>
>> ___________________________
>>
>> 1 78.2 0.261 AMU
>>
>>    77.9 0.261 FBK
>>
>>    76.8 0.221 DCU
>>
>> ___________________________
>>
>> 4 73.8 0.115 JXNU
>>
>> ___________________________
>>
>> 5 71.9 0.038 USAAR
>>
>>    71.1 0.014 CUNI
>>
>>    70.2 −0.020 LIG
>>
>> ___________________________
>>
>> − 68.6 −0.083 NO POST EDIT
>>
>> Table 35: EN-DE DA Human evaluation results showing average raw DA scores
>> (Ave %) and average standardized scores (Ave z), lines between systems
>> indicate clusters according to Wilcoxon rank-sum test at p-level p ≤ 0.05.
>>
>>  [/quote]
>>
>> Seems to indicate that human translation is better than machine
>> translation, but of course that doesn't guarantee that there isn't a better
>> translation program somewhere from pre 31 Dec 2015 that simply didn't
>> attend the conference.
>>
>> Still if human level translation existed in 2015, you would not expect to
>> read things like
>>
>> [quote]
>>
>> This steady improvement has been mainly driven by the massive migration
>> to the neural approach, which in 2016 allowed the winning system to achieve
>> impressive results
>>
>> [/quote]
>>
>> I don't believe there is a program that can justifiably claim "equal or
>> better average quality, as professional human translations" but proving a
>> negative is difficult. I suggest if there was such a program it would be
>> big news, not difficult to find, and conference findings would be markedly
>> different to those linked above.
>>
>> Not sure how much more a judge might want before deciding how to judge
>> the claim. Are there any more authoritative events or other event before
>> claim deadline of 31 Dec 2017? (Note program has to exist by 31 Dec 2015
>> and translations have to 'be of comparable cost and turnaround time'.)
>>
>>
>> The comparable cost and turnaround time requirement seems to me to
>> indicate that secret research would not qualify.
>>
>>
>> Seems like an obvious false to me.
>>
>> Regards
>>
>> Chris Randles
>>
>> (crandles 7886)
>>
>> Disclosure I hold -3603 in this claim
>>
>>
>>
>>
>