Ideosphere Forum
From owner-fx-discuss@ideosphere.com Sat Jan 20 16:39:16 2018
Received: from ideosphere.com (localhost [127.0.0.1])
	by ideosphere.com (8.13.8/8.13.8) with ESMTP id w0KLdGxk005438
	for <fx-discuss-outgoing@ideosphere.com>; Sat, 20 Jan 2018 16:39:16 -0500
Received: (from majordomo@localhost)
	by ideosphere.com (8.13.8/8.13.8/Submit) id w0KLdGRo005437
	for fx-discuss-outgoing; Sat, 20 Jan 2018 16:39:16 -0500
Received: from mail-ua0-f170.google.com (mail-ua0-f170.google.com [209.85.217.170])
	by ideosphere.com (8.13.8/8.13.8) with ESMTP id w0KLdFSG005433
	for <fx-discuss@ideosphere.com>; Sat, 20 Jan 2018 16:39:16 -0500
Received: by mail-ua0-f170.google.com with SMTP id p12so848239uad.0
        for <fx-discuss@ideosphere.com>; Sat, 20 Jan 2018 13:39:16 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to;
        bh=+dGmkbPWZYZ6/RMEo+Dd3QmwN9z0iMh4LeSH+Qbdyhg=;
        b=CXbqQXRdklJQYCbWe5hXHt2htxD81fUCBbYhZcdyrNI/82W3DtHwNNkWzGWjo2NTTl
         wC8wFEAxCYcEWE9v1joZbgecuYzpujecrLWuoPAZ5kQxKwHcFXoGiClvszqlV2OhNLbJ
         aYeNJyiuDLhVUBZQs8haVky4oyw6ulxfuCDlAzCneqytJ/XLOyw23WFXYhcW9SUgYi9I
         sMcr8GZDZnGFxbbR2wBBm1MQveoRQcQ0do9H9hpAYNhRTP9EzxXQjhulq3VtFV7YD9c/
         Sh/N8pMd/8wvD+ZsoCQirhU/FrZFuriD+T5n9SpU5RmySKrVJSEY5RHWnOORGC9Qs/cb
         XUsA==
X-Gm-Message-State: AKwxyteU8z/qyx01Hbp1pr/yHBxH4x1Gy2WxnmShoen/1Xh/RpkGI+1r
	ZSqfcuqN8L2rskro5g7skYtD9dttFr31Op2rTsPZ9g==
X-Google-Smtp-Source: AH8x2242g9c8Fnaq6DxRZ355x8TherYY4td0kwTqpikGQR6crQlLxU7ohY5s8XQSmAQmoaQ6LuSx1tI4arMccTpZH8M=
X-Received: by 10.176.76.66 with SMTP id d2mr2007461uag.176.1516484354648;
 Sat, 20 Jan 2018 13:39:14 -0800 (PST)
MIME-Version: 1.0
References: <862447574.1932061.1515241909823.JavaMail.open-xchange@oxbe25.tb.ukmail.iss.as9143.net>
 <556361200.2395480.1516457867962.JavaMail.open-xchange@oxbe21.tb.ukmail.iss.as9143.net>
In-Reply-To: <556361200.2395480.1516457867962.JavaMail.open-xchange@oxbe21.tb.ukmail.iss.as9143.net>
From: Neal Gafter <neal@gafter.com>
Date: Sat, 20 Jan 2018 21:39:03 +0000
Message-ID: <CAD=nd8cYwgnVeOmjHDsRSbtO0_=iEG-K5nuC4wZfdnhL-2gNVA@mail.gmail.com>
Subject: fx-discuss: =?UTF-8?Q?Re=3A_fx=2Ddiscuss=3A_Fwd=3A_FX_Claim=3A_Tran_=E2=80=93_Machine_tran?=
	=?UTF-8?Q?slation_by_2015?=
To: fx-discuss@ideosphere.com
Content-Type: multipart/alternative; boundary="f40304361566442ec305633c07e2"
Sender: owner-fx-discuss@ideosphere.com
Precedence: bulk
Reply-To: fx-discuss@ideosphere.com

 
 I am still around. I will start looking at this in a few days.

On Sat, Jan 20, 2018 at 9:18 AM chrisran.bma e-mail <chrisran.bma@virgin.net>
wrote:

> Hi,
>
> The due date for this claim 2017/12/31 has passed.
>
> Does anyone think or preferably have evidence to suggest this claim is
> true or false?
>
> Is the judge, Neal Gafter aka loophole still around?
>
>
>
> I have previously written things like:
>
>
> Findings of the 2017 Conference on Machine Translation (WMT17)
>
> http://www.statmt.org/wmt17/pdf/WMT17.pdf
>
> (September 7-8, 2017)
>
> and
>
> Findings of the 2016 Conference on Machine Translation (WMT16)
>
> http://www.aclweb.org/anthology/W16-2301
>
> (11-12 August 2016)
>
> Perhaps the following looks like what we want:
>
> [quote]
>
> 5.5.2 Human evaluation results
>
> Table 35 includes DA results for English-German and Table 36 shows results
> for German-English APE systems. Clusters are identified by grouping systems
> together according to which systems significantly outperform all others in
> lower ranking clusters, according to Wilcoxon rank-sum test.
>
> # Ave % Ave z System
>
> ___________________________
>
> − 84.8 0.520 HUMAN POST EDIT
>
> ___________________________
>
> 1 78.2 0.261 AMU
>
>    77.9 0.261 FBK
>
>    76.8 0.221 DCU
>
> ___________________________
>
> 4 73.8 0.115 JXNU
>
> ___________________________
>
> 5 71.9 0.038 USAAR
>
>    71.1 0.014 CUNI
>
>    70.2 −0.020 LIG
>
> ___________________________
>
> − 68.6 −0.083 NO POST EDIT
>
> Table 35: EN-DE DA Human evaluation results showing average raw DA scores
> (Ave %) and average standardized scores (Ave z), lines between systems
> indicate clusters according to Wilcoxon rank-sum test at p-level p ≤ 0.05.
>
>  [/quote]
>
> Seems to indicate that human translation is better than machine
> translation, but of course that doesn't guarantee that there isn't a better
> translation program somewhere from pre 31 Dec 2015 that simply didn't
> attend the conference.
>
> Still if human level translation existed in 2015, you would not expect to
> read things like
>
> [quote]
>
> This steady improvement has been mainly driven by the massive migration to
> the neural approach, which in 2016 allowed the winning system to achieve
> impressive results
>
> [/quote]
>
> I don't believe there is a program that can justifiably claim "equal or
> better average quality, as professional human translations" but proving a
> negative is difficult. I suggest if there was such a program it would be
> big news, not difficult to find, and conference findings would be markedly
> different to those linked above.
>
> Not sure how much more a judge might want before deciding how to judge the
> claim. Are there any more authoritative events or other event before claim
> deadline of 31 Dec 2017? (Note program has to exist by 31 Dec 2015 and
> translations have to 'be of comparable cost and turnaround time'.)
>
>
> The comparable cost and turnaround time requirement seems to me to
> indicate that secret research would not qualify.
>
>
> Seems like an obvious false to me.
>
> Regards
>
> Chris Randles
>
> (crandles 7886)
>
> Disclosure I hold -3603 in this claim
>
>
>
>

Back to regular message display

All trademarks, copyrights, and messages on this page are owned by their respective owners.
Forum: Copyright (c) 2000-2001 Javien Inc All rights reserved. Distributed under the GPL