你的位置:首页 > 信息动态 > 新闻中心
信息动态
联系我们

flink cdc 支持1.14.2 版本记录

2021/12/30 23:37:47

https://ververica.github.io/flink-cdc-connectors 2.1.1是居于flink 1.13.3 开发的,使用了flink shaded guava18等类,在1.14.2会有如下错误:

WARN  org.apache.flink.runtime.taskmanager.Task                    [] - Source: TableSourceScan(table=[[default_catalog, default_database, shipments]], fields=[shipment_id, order_id, origin, destination, is_arrived]) (1/1)#366 (08b8e7715113f27e5f5342ca9d554576) switched from INITIALIZING to FAILED with failure cause: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/util/concurrent/ThreadFactoryBuilder

查看flink jar,可以看到已经改用30版本了,如下图:
在这里插入图片描述

简单记录下改为依赖flink 1.14.2 需要修改的地方。

  • 将org.apache.flink.shaded.guava18 改为org.apache.flink.shaded.guava30
  • scala.binary.version 版本改为2.12
  • 去掉所有blink 依赖(1.14.2 已经去掉了blink jar,参考https://issues.apache.org/jira/browse/FLINK-22879)
  • JsonOptions类不存在了,需要做对应修改

JsonOptions具体修改如下:

   public static final ConfigOption<Boolean> IGNORE_PARSE_ERRORS = JsonOptions.IGNORE_PARSE_ERRORS;

    public static final ConfigOption<String> TIMESTAMP_FORMAT = JsonOptions.TIMESTAMP_FORMAT;

改为

   public static final ConfigOption<Boolean> IGNORE_PARSE_ERRORS = JsonFormatOptions.IGNORE_PARSE_ERRORS;

    public static final ConfigOption<String> TIMESTAMP_FORMAT = JsonFormatOptions.TIMESTAMP_FORMAT;
 TimestampFormat timestampFormat = JsonOptions.getTimestampFormat(formatOptions);

改为

        TimestampFormat timestampFormat = JsonFormatOptionsUtil.getTimestampFormat(formatOptions);