Pipeline environment for batch processing
4 months ago
Version
Change
Description
4.3
CHANGED
Updated on 2025-11-25
An existing pipeline environment for batch processing has been changed: batch-4.3 for batch environment with Spark 3.4.4 and Scala 2.13.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.5.20ch.qos.logbacklogback-core1.5.20com.amazonawsaws-java-sdk-bundle1.12.781com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.132.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.130.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.github.java-diff-utilsjava-diff-utils4.15io.nettynetty-all4.1.126.Finalio.nettynetty-buffer4.1.126.Finalio.nettynetty-codec4.1.126.Finalio.nettynetty-codec-dns4.1.126.Finalio.nettynetty-codec-http4.1.126.Finalio.nettynetty-codec-http24.1.126.Finalio.nettynetty-codec-socks4.1.126.Finalio.nettynetty-common4.1.126.Finalio.nettynetty-handler4.1.126.Finalio.nettynetty-handler-proxy4.1.126.Finalio.nettynetty-resolver4.1.126.Finalio.nettynetty-resolver-dns4.1.126.Finalio.nettynetty-transport4.1.126.Finalio.nettynetty-transport-classes-epoll4.1.126.Finalio.nettynetty-transport-classes-kqueue4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-unix-common4.1.126.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.java.dev.jnajna5.9.0net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-avro_2.133.4.4org.apache.sparkspark-catalyst_2.133.4.4org.apache.sparkspark-core_2.133.4.4org.apache.sparkspark-graphx_2.133.4.4org.apache.sparkspark-hive-thriftserver_2.133.4.4org.apache.sparkspark-hive_2.133.4.4org.apache.sparkspark-kubernetes_2.133.4.4org.apache.sparkspark-kvstore_2.133.4.4org.apache.sparkspark-launcher_2.133.4.4org.apache.sparkspark-mesos_2.133.4.4org.apache.sparkspark-mllib-local_2.133.4.4org.apache.sparkspark-mllib_2.133.4.4org.apache.sparkspark-network-common_2.133.4.4org.apache.sparkspark-network-shuffle_2.133.4.4org.apache.sparkspark-repl_2.133.4.4org.apache.sparkspark-sketch_2.133.4.4org.apache.sparkspark-sql_2.133.4.4org.apache.sparkspark-streaming_2.133.4.4org.apache.sparkspark-tags_2.133.4.4org.apache.sparkspark-unsafe_2.133.4.4org.apache.sparkspark-yarn_2.133.4.4org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.jlinejline3.21.0org.jlinejline-builtins3.21.0org.jlinejline-reader3.21.0org.jlinejline-remote-ssh3.21.0org.jlinejline-remote-telnet3.21.0org.jlinejline-style3.21.0org.jlinejline-terminal3.21.0org.jlinejline-terminal-jansi3.21.0org.jlinejline-terminal-jna3.21.0org.joddjodd-core5.0.13org.json4sjson4s-ast_2.133.7.0-M11org.json4sjson4s-core_2.133.7.0-M11org.json4sjson4s-jackson_2.133.7.0-M11org.json4sjson4s-scalap_2.133.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.13.16org.scala-langscala-reflect2.13.16org.scala-lang.modulesscala-collection-compat_2.132.7.0org.scala-lang.modulesscala-parallel-collections_2.131.0.4org.scala-lang.modulesscala-parser-combinators_2.132.1.1org.scala-lang.modulesscala-xml_2.132.1.0org.scalanlpbreeze-macros_2.132.1.0org.scalanlpbreeze_2.132.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.132.8.0org.typelevelcats-kernel_2.132.8.0org.typelevelspire-macros_2.130.18.0org.typelevelspire-platform_2.130.18.0org.typelevelspire-util_2.130.18.0org.typelevelspire_2.130.18.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.3 for batch environment with Spark 3.4.4 and Scala 2.13.
The following lists all dependencies provided by this new pipeline environment:
4.3
DEPRECATED
Deprecated on 2026-01-01
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 12 months:
batch-4.3 for batch environment with Spark 3.4.4 and Scala 2.13.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 12 months:
batch-4.3 for batch environment with Spark 3.4.4 and Scala 2.13.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
5.0
ADDED
Added on 2025-11-25
A new pipeline environment for batch processing has been introduced:
batch-5.0 for batch environment with Spark 4.0.1.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.epfl.scalabsp4j2.1.0-M5ch.qos.logbacklogback-classic1.5.20ch.qos.logbacklogback-core1.5.20com.amazonawsaws-java-sdk-core1.12.781com.amazonawsaws-java-sdk-sts1.12.781com.clearspring.analyticsstream2.9.8com.esotericsoftwarekryo-shaded4.0.3com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.18.2com.fasterxml.jackson.corejackson-core2.18.2com.fasterxml.jackson.corejackson-databind2.18.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.18.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.18.2com.fasterxml.jackson.modulejackson-module-scala_2.132.18.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.javaparserjavaparser-core3.2.12com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.6-9com.github.wendykierpJTransforms3.1com.google.androidannotations4.1.1.4com.google.api.grpcproto-google-common-protos2.41.0com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.11.0com.google.crypto.tinktink1.16.0com.google.errorproneerror_prone_annotations2.36.0com.google.flatbuffersflatbuffers-java24.3.25com.google.guavafailureaccess1.0.2com.google.guavaguava33.4.0-jrecom.google.guavalistenablefuture9999.0-empty-to-avoid-conflict-with-guavacom.google.j2objcj2objc-annotations3.0.0com.google.protobufprotobuf-java-util4.29.3com.ibm.icuicu4j76.1com.lihaoyiammonite-compiler-interface_2.13.163.0.1com.lihaoyiammonite-compiler_2.13.163.0.1com.lihaoyiammonite-interp-api_2.13.163.0.1com.lihaoyiammonite-interp_2.13.163.0.1com.lihaoyiammonite-repl-api_2.13.163.0.1com.lihaoyiammonite-repl_2.13.163.0.1com.lihaoyiammonite-runtime_2.13.163.0.1com.lihaoyiammonite-terminal_2.133.0.1com.lihaoyiammonite-util_2.133.0.1com.lihaoyiammonite_2.13.163.0.1com.lihaoyifansi_2.130.5.0com.lihaoyifastparse_2.133.1.1com.lihaoyigeny_2.131.1.1com.lihaoyimainargs_2.130.7.5com.lihaoyios-lib_2.130.11.2com.lihaoyipprint_2.130.9.0com.lihaoyirequests_2.130.9.0com.lihaoyiscalaparse_2.133.1.1com.lihaoyisourcecode_2.130.4.3-M1com.lihaoyiujson_2.134.0.2com.lihaoyiupack_2.134.0.2com.lihaoyiupickle-core_2.134.0.2com.lihaoyiupickle-implicits_2.134.0.2com.lihaoyiupickle_2.134.0.2com.ningcompress-lzf1.1.2com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime4.1.2com.tdunningjson1.8com.thesamet.scalapblenses_2.130.11.17com.thesamet.scalapbscalapb-runtime_2.130.11.17com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.130.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.9.0commons-codeccommons-codec1.17.2commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.18.0commons-langcommons-lang2.6commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor2.0.2io.dropwizard.metricsmetrics-core4.2.30io.dropwizard.metricsmetrics-graphite4.2.30io.dropwizard.metricsmetrics-jmx4.2.30io.dropwizard.metricsmetrics-json4.2.30io.dropwizard.metricsmetrics-jvm4.2.30io.fabric8kubernetes-client7.1.0io.fabric8kubernetes-client-api7.1.0io.fabric8kubernetes-httpclient-vertx7.1.0io.fabric8kubernetes-model-admissionregistration7.1.0io.fabric8kubernetes-model-apiextensions7.1.0io.fabric8kubernetes-model-apps7.1.0io.fabric8kubernetes-model-autoscaling7.1.0io.fabric8kubernetes-model-batch7.1.0io.fabric8kubernetes-model-certificates7.1.0io.fabric8kubernetes-model-common7.1.0io.fabric8kubernetes-model-coordination7.1.0io.fabric8kubernetes-model-core7.1.0io.fabric8kubernetes-model-discovery7.1.0io.fabric8kubernetes-model-events7.1.0io.fabric8kubernetes-model-extensions7.1.0io.fabric8kubernetes-model-flowcontrol7.1.0io.fabric8kubernetes-model-gatewayapi7.1.0io.fabric8kubernetes-model-metrics7.1.0io.fabric8kubernetes-model-networking7.1.0io.fabric8kubernetes-model-node7.1.0io.fabric8kubernetes-model-policy7.1.0io.fabric8kubernetes-model-rbac7.1.0io.fabric8kubernetes-model-resource7.1.0io.fabric8kubernetes-model-scheduling7.1.0io.fabric8kubernetes-model-storageclass7.1.0io.fabric8zjsonpatch7.1.0io.get-coursierclass-path-util_2.130.1.4io.get-coursierdependency-interface_2.130.2.3io.get-coursierdependency_2.130.2.3io.get-coursierinterface1.0.19io.github.java-diff-utilsjava-diff-utils4.15io.jsonwebtokenjjwt-api0.12.6io.jsonwebtokenjjwt-impl0.12.6io.jsonwebtokenjjwt-jackson0.12.6io.nettynetty-all4.1.126.Finalio.nettynetty-buffer4.1.126.Finalio.nettynetty-codec4.1.126.Finalio.nettynetty-codec-dns4.1.126.Finalio.nettynetty-codec-http4.1.126.Finalio.nettynetty-codec-http24.1.126.Finalio.nettynetty-codec-socks4.1.126.Finalio.nettynetty-common4.1.126.Finalio.nettynetty-handler4.1.126.Finalio.nettynetty-handler-proxy4.1.126.Finalio.nettynetty-resolver4.1.126.Finalio.nettynetty-resolver-dns4.1.126.Finalio.nettynetty-tcnative-boringssl-static2.0.73.Finalio.nettynetty-tcnative-boringssl-static2.0.73.Finalio.nettynetty-tcnative-boringssl-static2.0.73.Finalio.nettynetty-tcnative-boringssl-static2.0.73.Finalio.nettynetty-tcnative-boringssl-static2.0.73.Finalio.nettynetty-tcnative-classes2.0.73.Finalio.nettynetty-transport4.1.126.Finalio.nettynetty-transport-classes-epoll4.1.126.Finalio.nettynetty-transport-classes-kqueue4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-unix-common4.1.126.Finalio.vertxvertx-auth-common4.5.12io.vertxvertx-core4.5.12io.vertxvertx-web-client4.5.12io.vertxvertx-web-common4.5.12jakarta.activationjakarta.activation-api2.1.3jakarta.annotationjakarta.annotation-api2.1.1jakarta.injectjakarta.inject-api2.0.1jakarta.servletjakarta.servlet-api5.0.0jakarta.validationjakarta.validation-api3.0.2jakarta.ws.rsjakarta.ws.rs-api3.0.0jakarta.xml.bindjakarta.xml.bind-api4.0.2javax.jdojdo-api3.0.1javax.servletjavax.servlet-api4.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.13.0net.openhftzero-allocation-hashing0.26ea0net.razorvinepickle1.5net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.9net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.13.1org.antlrST44.0.4org.apache.arrowarrow-format18.1.0org.apache.arrowarrow-memory-core18.1.0org.apache.arrowarrow-memory-netty18.1.0org.apache.arrowarrow-memory-netty-buffer-patch18.1.0org.apache.arrowarrow-vector18.1.0org.apache.avroavro1.12.0org.apache.avroavro-ipc1.12.0org.apache.avroavro-mapred1.12.0org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.12.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.13.0org.apache.curatorcurator-client5.7.1org.apache.curatorcurator-framework5.7.1org.apache.curatorcurator-recipes5.7.1org.apache.datasketchesdatasketches-java6.1.1org.apache.datasketchesdatasketches-memory3.0.2org.apache.derbyderby10.16.1.1org.apache.derbyderbyshared10.16.1.1org.apache.derbyderbytools10.16.1.1org.apache.hadoophadoop-annotations3.4.1org.apache.hadoophadoop-auth3.4.1org.apache.hadoophadoop-aws3.4.1org.apache.hadoophadoop-client3.4.1org.apache.hadoophadoop-client-api3.4.1org.apache.hadoophadoop-client-runtime3.4.1org.apache.hadoophadoop-common3.4.1org.apache.hadoophadoop-hdfs-client3.4.1org.apache.hadoophadoop-mapreduce-client-common3.4.1org.apache.hadoophadoop-mapreduce-client-core3.4.1org.apache.hadoophadoop-mapreduce-client-jobclient3.4.1org.apache.hadoophadoop-yarn-api3.4.1org.apache.hadoophadoop-yarn-client3.4.1org.apache.hadoophadoop-yarn-common3.4.1org.apache.hivehive-beeline2.3.10org.apache.hivehive-cli2.3.10org.apache.hivehive-common2.3.10org.apache.hivehive-exec2.3.10org.apache.hivehive-jdbc2.3.10org.apache.hivehive-metastore2.3.10org.apache.hivehive-serde2.3.10org.apache.hivehive-service-rpc4.0.0org.apache.hivehive-shims2.3.10org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.10org.apache.hive.shimshive-shims-common2.3.10org.apache.hive.shimshive-shims-scheduler2.3.10org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.3org.apache.logging.log4jlog4j-api2.24.3org.apache.logging.log4jlog4j-core2.24.3org.apache.logging.log4jlog4j-layout-template-json2.24.3org.apache.logging.log4jlog4j-to-slf4j2.24.3org.apache.orcorc-core2.1.3org.apache.orcorc-format1.1.1org.apache.orcorc-mapreduce2.1.3org.apache.orcorc-shims2.1.3org.apache.parquetparquet-column1.15.2org.apache.parquetparquet-common1.15.2org.apache.parquetparquet-encoding1.15.2org.apache.parquetparquet-format-structures1.15.2org.apache.parquetparquet-hadoop1.15.2org.apache.parquetparquet-jackson1.15.2org.apache.sparkspark-avro_2.134.0.1org.apache.sparkspark-catalyst_2.134.0.1org.apache.sparkspark-common-utils_2.134.0.1org.apache.sparkspark-connect-client-jvm_2.134.0.1org.apache.sparkspark-connect-common_2.134.0.1org.apache.sparkspark-connect_2.134.0.1org.apache.sparkspark-core_2.134.0.1org.apache.sparkspark-graphx_2.134.0.1org.apache.sparkspark-hive-thriftserver_2.134.0.1org.apache.sparkspark-hive_2.134.0.1org.apache.sparkspark-kubernetes_2.134.0.1org.apache.sparkspark-kvstore_2.134.0.1org.apache.sparkspark-launcher_2.134.0.1org.apache.sparkspark-mllib-local_2.134.0.1org.apache.sparkspark-mllib_2.134.0.1org.apache.sparkspark-network-common_2.134.0.1org.apache.sparkspark-network-shuffle_2.134.0.1org.apache.sparkspark-repl_2.134.0.1org.apache.sparkspark-sketch_2.134.0.1org.apache.sparkspark-sql-api_2.134.0.1org.apache.sparkspark-sql_2.134.0.1org.apache.sparkspark-streaming_2.134.0.1org.apache.sparkspark-tags_2.134.0.1org.apache.sparkspark-unsafe_2.134.0.1org.apache.sparkspark-variant_2.134.0.1org.apache.sparkspark-yarn_2.134.0.1org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.16.0org.apache.ws.xmlschemaxmlschema-core2.3.1org.apache.xbeanxbean-asm9-shaded4.26org.apache.yetusaudience-annotations0.12.0org.apache.zookeeperzookeeper3.9.3org.apache.zookeeperzookeeper-jute3.9.3org.bouncycastlebcprov-jdk18on1.80org.checkerframeworkchecker-qual3.43.0org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.mojoanimal-sniffer-annotations1.24org.codehaus.woodstoxstax2-api4.2.2org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.eclipse.lsp4jorg.eclipse.lsp4j.generator0.20.1org.eclipse.lsp4jorg.eclipse.lsp4j.jsonrpc0.20.1org.eclipse.xtendorg.eclipse.xtend.lib2.28.0org.eclipse.xtendorg.eclipse.xtend.lib.macro2.28.0org.eclipse.xtextorg.eclipse.xtext.xbase.lib2.28.0org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api3.0.6org.glassfish.hk2hk2-locator3.0.6org.glassfish.hk2hk2-utils3.0.6org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged3.0.6org.glassfish.jaxbjaxb-core4.0.5org.glassfish.jaxbjaxb-runtime4.0.5org.glassfish.jersey.containersjersey-container-servlet3.0.16org.glassfish.jersey.containersjersey-container-servlet-core3.0.16org.glassfish.jersey.corejersey-client3.0.16org.glassfish.jersey.corejersey-common3.0.16org.glassfish.jersey.corejersey-server3.0.16org.glassfish.jersey.injectjersey-hk23.0.16org.javassistjavassist3.30.2-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.jlinejansi-core3.27.1org.jlinejline3.27.1org.jlinejline-builtins3.27.1org.jlinejline-native3.27.1org.jlinejline-reader3.27.1org.jlinejline-remote-ssh3.27.1org.jlinejline-remote-telnet3.27.1org.jlinejline-style3.27.1org.jlinejline-terminal3.27.1org.jlinejline-terminal-ffm3.27.1org.jlinejline-terminal-jansi3.27.1org.jlinejline-terminal-jna3.27.1org.jlinejline-terminal-jni3.27.1org.joddjodd-core5.0.13org.json4sjson4s-ast_2.134.0.7org.json4sjson4s-core_2.134.0.7org.json4sjson4s-jackson-core_2.134.0.7org.json4sjson4s-jackson_2.134.0.7org.json4sjson4s-scalap_2.134.0.7org.lz4lz4-java1.8.0org.objenesisobjenesis3.3org.roaringbitmapRoaringBitmap1.3.0org.rocksdbrocksdbjni9.8.4org.scala-langscala-library2.13.16org.scala-langscala-reflect2.13.16org.scala-lang.modulesscala-collection-compat_2.132.7.0org.scala-lang.modulesscala-parallel-collections_2.131.2.0org.scala-lang.modulesscala-parser-combinators_2.132.4.0org.scala-lang.modulesscala-xml_2.132.3.0org.scalametacommon_2.134.9.7org.scalanlpbreeze-macros_2.132.1.0org.scalanlpbreeze_2.132.1.0org.slf4jjcl-over-slf4j2.0.16org.slf4jjul-to-slf4j2.0.16org.slf4jlog4j-over-slf4j2.0.16org.slf4jslf4j-api2.0.16org.snakeyamlsnakeyaml-engine2.9org.threetenthreeten-extra1.8.0org.tpolecattypename_2.131.1.0org.tukaanixz1.10org.typelevelalgebra_2.132.8.0org.typelevelcats-kernel_2.132.8.0org.typelevelspire-macros_2.130.18.0org.typelevelspire-platform_2.130.18.0org.typelevelspire-util_2.130.18.0org.typelevelspire_2.130.18.0org.wildfly.commonwildfly-common1.5.4.Finalorg.wildfly.opensslwildfly-openssl2.2.5.Finalorg.wildfly.opensslwildfly-openssl-java2.2.5.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_642.2.2.SP01org.wildfly.opensslwildfly-openssl-macosx-x86_642.2.2.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_642.2.2.Finalorg.xerial.snappysnappy-java1.1.10.7org.yamlsnakeyaml2.3orooro2.0.8pl.edu.icmJLargeArrays1.5software.amazon.awssdkbundle2.24.6staxstax-api1.0.1
A new pipeline environment for batch processing has been introduced:
batch-5.0 for batch environment with Spark 4.0.1.
The following lists all dependencies provided by this new pipeline environment:
4.3
ADDED
Added on 2025-10-28
A new pipeline environment for batch processing has been introduced:
batch-4.3 for batch environment with Spark 3.4.4 and Scala 2.13.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.4.14ch.qos.logbacklogback-core1.4.14com.amazonawsaws-java-sdk-bundle1.12.781com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.132.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.130.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.github.java-diff-utilsjava-diff-utils4.15io.nettynetty-all4.1.126.Finalio.nettynetty-buffer4.1.126.Finalio.nettynetty-codec4.1.126.Finalio.nettynetty-codec-dns4.1.126.Finalio.nettynetty-codec-http4.1.126.Finalio.nettynetty-codec-http24.1.126.Finalio.nettynetty-codec-socks4.1.126.Finalio.nettynetty-common4.1.126.Finalio.nettynetty-handler4.1.126.Finalio.nettynetty-handler-proxy4.1.126.Finalio.nettynetty-resolver4.1.126.Finalio.nettynetty-resolver-dns4.1.126.Finalio.nettynetty-transport4.1.126.Finalio.nettynetty-transport-classes-epoll4.1.126.Finalio.nettynetty-transport-classes-kqueue4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-unix-common4.1.126.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.java.dev.jnajna5.9.0net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-avro_2.133.4.4org.apache.sparkspark-catalyst_2.133.4.4org.apache.sparkspark-core_2.133.4.4org.apache.sparkspark-graphx_2.133.4.4org.apache.sparkspark-hive-thriftserver_2.133.4.4org.apache.sparkspark-hive_2.133.4.4org.apache.sparkspark-kubernetes_2.133.4.4org.apache.sparkspark-kvstore_2.133.4.4org.apache.sparkspark-launcher_2.133.4.4org.apache.sparkspark-mesos_2.133.4.4org.apache.sparkspark-mllib-local_2.133.4.4org.apache.sparkspark-mllib_2.133.4.4org.apache.sparkspark-network-common_2.133.4.4org.apache.sparkspark-network-shuffle_2.133.4.4org.apache.sparkspark-repl_2.133.4.4org.apache.sparkspark-sketch_2.133.4.4org.apache.sparkspark-sql_2.133.4.4org.apache.sparkspark-streaming_2.133.4.4org.apache.sparkspark-tags_2.133.4.4org.apache.sparkspark-unsafe_2.133.4.4org.apache.sparkspark-yarn_2.133.4.4org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.jlinejline3.21.0org.jlinejline-builtins3.21.0org.jlinejline-reader3.21.0org.jlinejline-remote-ssh3.21.0org.jlinejline-remote-telnet3.21.0org.jlinejline-style3.21.0org.jlinejline-terminal3.21.0org.jlinejline-terminal-jansi3.21.0org.jlinejline-terminal-jna3.21.0org.joddjodd-core5.0.13org.json4sjson4s-ast_2.133.7.0-M11org.json4sjson4s-core_2.133.7.0-M11org.json4sjson4s-jackson_2.133.7.0-M11org.json4sjson4s-scalap_2.133.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.13.16org.scala-langscala-reflect2.13.16org.scala-lang.modulesscala-collection-compat_2.132.7.0org.scala-lang.modulesscala-parallel-collections_2.131.0.4org.scala-lang.modulesscala-parser-combinators_2.132.1.1org.scala-lang.modulesscala-xml_2.132.1.0org.scalanlpbreeze-macros_2.132.1.0org.scalanlpbreeze_2.132.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.132.8.0org.typelevelcats-kernel_2.132.8.0org.typelevelspire-macros_2.130.18.0org.typelevelspire-platform_2.130.18.0org.typelevelspire-util_2.130.18.0org.typelevelspire_2.130.18.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
A new pipeline environment for batch processing has been introduced:
batch-4.3 for batch environment with Spark 3.4.4 and Scala 2.13.
The following lists all dependencies provided by this new pipeline environment:
4.0
CHANGED
Updated on 2025-09-29
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.4.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.3.14ch.qos.logbacklogback-core1.3.14com.amazonawsaws-java-sdk-bundle1.12.781com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.126.Finalio.nettynetty-buffer4.1.126.Finalio.nettynetty-codec4.1.126.Finalio.nettynetty-codec-dns4.1.126.Finalio.nettynetty-codec-http4.1.126.Finalio.nettynetty-codec-http24.1.126.Finalio.nettynetty-codec-socks4.1.126.Finalio.nettynetty-common4.1.126.Finalio.nettynetty-handler4.1.126.Finalio.nettynetty-handler-proxy4.1.126.Finalio.nettynetty-resolver4.1.126.Finalio.nettynetty-resolver-dns4.1.126.Finalio.nettynetty-transport4.1.126.Finalio.nettynetty-transport-classes-epoll4.1.126.Finalio.nettynetty-transport-classes-kqueue4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-unix-common4.1.126.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-avro_2.123.4.4org.apache.sparkspark-catalyst_2.123.4.4org.apache.sparkspark-core_2.123.4.4org.apache.sparkspark-graphx_2.123.4.4org.apache.sparkspark-hive-thriftserver_2.123.4.4org.apache.sparkspark-hive_2.123.4.4org.apache.sparkspark-kubernetes_2.123.4.4org.apache.sparkspark-kvstore_2.123.4.4org.apache.sparkspark-launcher_2.123.4.4org.apache.sparkspark-mesos_2.123.4.4org.apache.sparkspark-mllib-local_2.123.4.4org.apache.sparkspark-mllib_2.123.4.4org.apache.sparkspark-network-common_2.123.4.4org.apache.sparkspark-network-shuffle_2.123.4.4org.apache.sparkspark-repl_2.123.4.4org.apache.sparkspark-sketch_2.123.4.4org.apache.sparkspark-sql_2.123.4.4org.apache.sparkspark-streaming_2.123.4.4org.apache.sparkspark-tags_2.123.4.4org.apache.sparkspark-unsafe_2.123.4.4org.apache.sparkspark-yarn_2.123.4.4org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.4.
The following lists all dependencies provided by this new pipeline environment:
4.1
CHANGED
Updated on 2025-11-25
An existing pipeline environment for batch processing has been changed: batch-4.1 for batch environment with Spark 3.4.4.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.5.20ch.qos.logbacklogback-core1.5.20com.amazonawsaws-java-sdk-bundle1.12.781com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.126.Finalio.nettynetty-buffer4.1.126.Finalio.nettynetty-codec4.1.126.Finalio.nettynetty-codec-dns4.1.126.Finalio.nettynetty-codec-http4.1.126.Finalio.nettynetty-codec-http24.1.126.Finalio.nettynetty-codec-socks4.1.126.Finalio.nettynetty-common4.1.126.Finalio.nettynetty-handler4.1.126.Finalio.nettynetty-handler-proxy4.1.126.Finalio.nettynetty-resolver4.1.126.Finalio.nettynetty-resolver-dns4.1.126.Finalio.nettynetty-transport4.1.126.Finalio.nettynetty-transport-classes-epoll4.1.126.Finalio.nettynetty-transport-classes-kqueue4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-unix-common4.1.126.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-avro_2.123.4.4org.apache.sparkspark-catalyst_2.123.4.4org.apache.sparkspark-core_2.123.4.4org.apache.sparkspark-graphx_2.123.4.4org.apache.sparkspark-hive-thriftserver_2.123.4.4org.apache.sparkspark-hive_2.123.4.4org.apache.sparkspark-kubernetes_2.123.4.4org.apache.sparkspark-kvstore_2.123.4.4org.apache.sparkspark-launcher_2.123.4.4org.apache.sparkspark-mesos_2.123.4.4org.apache.sparkspark-mllib-local_2.123.4.4org.apache.sparkspark-mllib_2.123.4.4org.apache.sparkspark-network-common_2.123.4.4org.apache.sparkspark-network-shuffle_2.123.4.4org.apache.sparkspark-repl_2.123.4.4org.apache.sparkspark-sketch_2.123.4.4org.apache.sparkspark-sql_2.123.4.4org.apache.sparkspark-streaming_2.123.4.4org.apache.sparkspark-tags_2.123.4.4org.apache.sparkspark-unsafe_2.123.4.4org.apache.sparkspark-yarn_2.123.4.4org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.1 for batch environment with Spark 3.4.4.
The following lists all dependencies provided by this new pipeline environment:
4.1
DEPRECATED
Deprecated on 2025-12-01
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 12 months:
batch-4.1 for batch environment with Spark 3.4.4 and Scala 2.12.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 12 months:
batch-4.1 for batch environment with Spark 3.4.4 and Scala 2.12.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
4.1
CHANGED
Updated on 2025-09-29
An existing pipeline environment for batch processing has been changed: batch-4.1 for batch environment with Spark 3.4.4.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.4.14ch.qos.logbacklogback-core1.4.14com.amazonawsaws-java-sdk-bundle1.12.781com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.126.Finalio.nettynetty-buffer4.1.126.Finalio.nettynetty-codec4.1.126.Finalio.nettynetty-codec-dns4.1.126.Finalio.nettynetty-codec-http4.1.126.Finalio.nettynetty-codec-http24.1.126.Finalio.nettynetty-codec-socks4.1.126.Finalio.nettynetty-common4.1.126.Finalio.nettynetty-handler4.1.126.Finalio.nettynetty-handler-proxy4.1.126.Finalio.nettynetty-resolver4.1.126.Finalio.nettynetty-resolver-dns4.1.126.Finalio.nettynetty-transport4.1.126.Finalio.nettynetty-transport-classes-epoll4.1.126.Finalio.nettynetty-transport-classes-kqueue4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-epoll4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-kqueue4.1.126.Finalio.nettynetty-transport-native-unix-common4.1.126.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-avro_2.123.4.4org.apache.sparkspark-catalyst_2.123.4.4org.apache.sparkspark-core_2.123.4.4org.apache.sparkspark-graphx_2.123.4.4org.apache.sparkspark-hive-thriftserver_2.123.4.4org.apache.sparkspark-hive_2.123.4.4org.apache.sparkspark-kubernetes_2.123.4.4org.apache.sparkspark-kvstore_2.123.4.4org.apache.sparkspark-launcher_2.123.4.4org.apache.sparkspark-mesos_2.123.4.4org.apache.sparkspark-mllib-local_2.123.4.4org.apache.sparkspark-mllib_2.123.4.4org.apache.sparkspark-network-common_2.123.4.4org.apache.sparkspark-network-shuffle_2.123.4.4org.apache.sparkspark-repl_2.123.4.4org.apache.sparkspark-sketch_2.123.4.4org.apache.sparkspark-sql_2.123.4.4org.apache.sparkspark-streaming_2.123.4.4org.apache.sparkspark-tags_2.123.4.4org.apache.sparkspark-unsafe_2.123.4.4org.apache.sparkspark-yarn_2.123.4.4org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.1 for batch environment with Spark 3.4.4.
The following lists all dependencies provided by this new pipeline environment:
4.1
CHANGED
Updated on 2025-07-16
An existing pipeline environment for batch processing has been changed: batch-4.1 for batch environment with Spark 3.4.4.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.4.14ch.qos.logbacklogback-core1.4.14com.amazonawsaws-java-sdk-bundle1.12.781com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.118.Finalio.nettynetty-buffer4.1.118.Finalio.nettynetty-codec4.1.118.Finalio.nettynetty-codec-http4.1.118.Finalio.nettynetty-codec-http24.1.118.Finalio.nettynetty-codec-socks4.1.118.Finalio.nettynetty-common4.1.118.Finalio.nettynetty-handler4.1.118.Finalio.nettynetty-handler-proxy4.1.118.Finalio.nettynetty-resolver4.1.118.Finalio.nettynetty-transport4.1.118.Finalio.nettynetty-transport-classes-epoll4.1.118.Finalio.nettynetty-transport-classes-kqueue4.1.118.Finalio.nettynetty-transport-native-epoll4.1.118.Finalio.nettynetty-transport-native-epoll4.1.118.Finalio.nettynetty-transport-native-kqueue4.1.118.Finalio.nettynetty-transport-native-kqueue4.1.118.Finalio.nettynetty-transport-native-unix-common4.1.118.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-avro_2.123.4.4org.apache.sparkspark-catalyst_2.123.4.4org.apache.sparkspark-core_2.123.4.4org.apache.sparkspark-graphx_2.123.4.4org.apache.sparkspark-hive-thriftserver_2.123.4.4org.apache.sparkspark-hive_2.123.4.4org.apache.sparkspark-kubernetes_2.123.4.4org.apache.sparkspark-kvstore_2.123.4.4org.apache.sparkspark-launcher_2.123.4.4org.apache.sparkspark-mesos_2.123.4.4org.apache.sparkspark-mllib-local_2.123.4.4org.apache.sparkspark-mllib_2.123.4.4org.apache.sparkspark-network-common_2.123.4.4org.apache.sparkspark-network-shuffle_2.123.4.4org.apache.sparkspark-repl_2.123.4.4org.apache.sparkspark-sketch_2.123.4.4org.apache.sparkspark-sql_2.123.4.4org.apache.sparkspark-streaming_2.123.4.4org.apache.sparkspark-tags_2.123.4.4org.apache.sparkspark-unsafe_2.123.4.4org.apache.sparkspark-yarn_2.123.4.4org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.1 for batch environment with Spark 3.4.4.
The following lists all dependencies provided by this new pipeline environment:
4.0
DEPRECATED
Deprecated on 2025-07-01
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 12 months:
batch-4.0 for batch environment with Spark 3.4.4.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 12 months:
batch-4.0 for batch environment with Spark 3.4.4.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
4.1
ADDED
Added on 2025-05-28
A new pipeline environment for batch processing has been introduced:
batch-4.1 for batch environment with Spark 3.4.4 and Java 17.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.4.14ch.qos.logbacklogback-core1.4.14com.amazonawsaws-java-sdk-bundle1.12.781com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.118.Finalio.nettynetty-buffer4.1.118.Finalio.nettynetty-codec4.1.118.Finalio.nettynetty-codec-http4.1.118.Finalio.nettynetty-codec-http24.1.118.Finalio.nettynetty-codec-socks4.1.118.Finalio.nettynetty-common4.1.118.Finalio.nettynetty-handler4.1.118.Finalio.nettynetty-handler-proxy4.1.118.Finalio.nettynetty-resolver4.1.118.Finalio.nettynetty-transport4.1.118.Finalio.nettynetty-transport-classes-epoll4.1.118.Finalio.nettynetty-transport-classes-kqueue4.1.118.Finalio.nettynetty-transport-native-epoll4.1.118.Finalio.nettynetty-transport-native-epoll4.1.118.Finalio.nettynetty-transport-native-kqueue4.1.118.Finalio.nettynetty-transport-native-kqueue4.1.118.Finalio.nettynetty-transport-native-unix-common4.1.118.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-catalyst_2.123.4.4org.apache.sparkspark-core_2.123.4.4org.apache.sparkspark-graphx_2.123.4.4org.apache.sparkspark-hive-thriftserver_2.123.4.4org.apache.sparkspark-hive_2.123.4.4org.apache.sparkspark-kubernetes_2.123.4.4org.apache.sparkspark-kvstore_2.123.4.4org.apache.sparkspark-launcher_2.123.4.4org.apache.sparkspark-mesos_2.123.4.4org.apache.sparkspark-mllib-local_2.123.4.4org.apache.sparkspark-mllib_2.123.4.4org.apache.sparkspark-network-common_2.123.4.4org.apache.sparkspark-network-shuffle_2.123.4.4org.apache.sparkspark-repl_2.123.4.4org.apache.sparkspark-sketch_2.123.4.4org.apache.sparkspark-sql_2.123.4.4org.apache.sparkspark-streaming_2.123.4.4org.apache.sparkspark-tags_2.123.4.4org.apache.sparkspark-unsafe_2.123.4.4org.apache.sparkspark-yarn_2.123.4.4org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
A new pipeline environment for batch processing has been introduced:
batch-4.1 for batch environment with Spark 3.4.4 and Java 17.
The following lists all dependencies provided by this new pipeline environment:
4.0
CHANGED
Updated on 2025-03-31
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.4.
Please visit official Apache Spark documentation page to check Spark 3.4.4 release notes.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.3.14ch.qos.logbacklogback-core1.3.14com.amazonawsaws-java-sdk-bundle1.12.781com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.118.Finalio.nettynetty-buffer4.1.118.Finalio.nettynetty-codec4.1.118.Finalio.nettynetty-codec-http4.1.118.Finalio.nettynetty-codec-http24.1.118.Finalio.nettynetty-codec-socks4.1.118.Finalio.nettynetty-common4.1.118.Finalio.nettynetty-handler4.1.118.Finalio.nettynetty-handler-proxy4.1.118.Finalio.nettynetty-resolver4.1.118.Finalio.nettynetty-transport4.1.118.Finalio.nettynetty-transport-classes-epoll4.1.118.Finalio.nettynetty-transport-classes-kqueue4.1.118.Finalio.nettynetty-transport-native-epoll4.1.118.Finalio.nettynetty-transport-native-epoll4.1.118.Finalio.nettynetty-transport-native-kqueue4.1.118.Finalio.nettynetty-transport-native-kqueue4.1.118.Finalio.nettynetty-transport-native-unix-common4.1.118.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-catalyst_2.123.4.4org.apache.sparkspark-core_2.123.4.4org.apache.sparkspark-graphx_2.123.4.4org.apache.sparkspark-hive-thriftserver_2.123.4.4org.apache.sparkspark-hive_2.123.4.4org.apache.sparkspark-kubernetes_2.123.4.4org.apache.sparkspark-kvstore_2.123.4.4org.apache.sparkspark-launcher_2.123.4.4org.apache.sparkspark-mesos_2.123.4.4org.apache.sparkspark-mllib-local_2.123.4.4org.apache.sparkspark-mllib_2.123.4.4org.apache.sparkspark-network-common_2.123.4.4org.apache.sparkspark-network-shuffle_2.123.4.4org.apache.sparkspark-repl_2.123.4.4org.apache.sparkspark-sketch_2.123.4.4org.apache.sparkspark-sql_2.123.4.4org.apache.sparkspark-streaming_2.123.4.4org.apache.sparkspark-tags_2.123.4.4org.apache.sparkspark-unsafe_2.123.4.4org.apache.sparkspark-yarn_2.123.4.4org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.4.
Please visit official Apache Spark documentation page to check Spark 3.4.4 release notes.
The following lists all dependencies provided by this new pipeline environment:
3.0
REMOVED
Removed on 2025-02-25
The deprecated pipeline environment batch-3.0 for batch processing with Spark 2.4.7 has been removed.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
The deprecated pipeline environment batch-3.0 for batch processing with Spark 2.4.7 has been removed.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
2.1.0
REMOVED
Removed on 2025-02-25
The deprecated pipeline environment batch-2.1.0 for batch processing with Spark 2.4.2 has been removed.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
The deprecated pipeline environment batch-2.1.0 for batch processing with Spark 2.4.2 has been removed.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
2.0.0
REMOVED
Removed on 2025-02-25
The deprecated pipeline environment batch-2.0.0 for batch processing with Spark 2.4.2 has been removed.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
The deprecated pipeline environment batch-2.0.0 for batch processing with Spark 2.4.2 has been removed.
For more information on how to migrate to the latest pipeline environment, please refer to this migration guide.
4.0
CHANGED
Updated on 2025-02-25
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.3.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.3.14ch.qos.logbacklogback-core1.3.14com.amazonawsaws-java-sdk-bundle1.12.638com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.17.1commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.16.1commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.27io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.113.Finalio.nettynetty-buffer4.1.113.Finalio.nettynetty-codec4.1.113.Finalio.nettynetty-codec-http4.1.113.Finalio.nettynetty-codec-http24.1.113.Finalio.nettynetty-codec-socks4.1.113.Finalio.nettynetty-common4.1.113.Finalio.nettynetty-handler4.1.113.Finalio.nettynetty-handler-proxy4.1.113.Finalio.nettynetty-resolver4.1.113.Finalio.nettynetty-transport4.1.113.Finalio.nettynetty-transport-classes-epoll4.1.113.Finalio.nettynetty-transport-classes-kqueue4.1.113.Finalio.nettynetty-transport-native-epoll4.1.113.Finalio.nettynetty-transport-native-epoll4.1.113.Finalio.nettynetty-transport-native-kqueue4.1.113.Finalio.nettynetty-transport-native-kqueue4.1.113.Finalio.nettynetty-transport-native-unix-common4.1.113.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.4org.apache.avroavro-ipc1.11.4org.apache.avroavro-mapred1.11.4org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.27.1org.apache.commonscommons-configuration22.11.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.17.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.12.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-catalyst_2.123.4.3org.apache.sparkspark-core_2.123.4.3org.apache.sparkspark-graphx_2.123.4.3org.apache.sparkspark-hive-thriftserver_2.123.4.3org.apache.sparkspark-hive_2.123.4.3org.apache.sparkspark-kubernetes_2.123.4.3org.apache.sparkspark-kvstore_2.123.4.3org.apache.sparkspark-launcher_2.123.4.3org.apache.sparkspark-mesos_2.123.4.3org.apache.sparkspark-mllib-local_2.123.4.3org.apache.sparkspark-mllib_2.123.4.3org.apache.sparkspark-network-common_2.123.4.3org.apache.sparkspark-network-shuffle_2.123.4.3org.apache.sparkspark-repl_2.123.4.3org.apache.sparkspark-sketch_2.123.4.3org.apache.sparkspark-sql_2.123.4.3org.apache.sparkspark-streaming_2.123.4.3org.apache.sparkspark-tags_2.123.4.3org.apache.sparkspark-unsafe_2.123.4.3org.apache.sparkspark-yarn_2.123.4.3org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core4.0.5org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.3.
The following lists all dependencies provided by this new pipeline environment:
4.0
CHANGED
Updated on 2024-06-26
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.3.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.3.14ch.qos.logbacklogback-core1.3.14com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.15commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.11.0commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.21io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.87.Finalio.nettynetty-buffer4.1.87.Finalio.nettynetty-codec4.1.87.Finalio.nettynetty-codec-http4.1.87.Finalio.nettynetty-codec-http24.1.87.Finalio.nettynetty-codec-socks4.1.87.Finalio.nettynetty-common4.1.87.Finalio.nettynetty-handler4.1.87.Finalio.nettynetty-handler-proxy4.1.87.Finalio.nettynetty-resolver4.1.87.Finalio.nettynetty-transport4.1.87.Finalio.nettynetty-transport-classes-epoll4.1.87.Finalio.nettynetty-transport-classes-kqueue4.1.87.Finalio.nettynetty-transport-native-epoll4.1.87.Finalio.nettynetty-transport-native-epoll4.1.87.Finalio.nettynetty-transport-native-kqueue4.1.87.Finalio.nettynetty-transport-native-kqueue4.1.87.Finalio.nettynetty-transport-native-unix-common4.1.87.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.3org.apache.avroavro-ipc1.11.3org.apache.avroavro-mapred1.11.3org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.22org.apache.commonscommons-configuration22.8.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.12.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.10.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.7org.apache.orcorc-mapreduce1.8.7org.apache.orcorc-shims1.8.7org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-catalyst_2.123.4.3org.apache.sparkspark-core_2.123.4.3org.apache.sparkspark-graphx_2.123.4.3org.apache.sparkspark-hive-thriftserver_2.123.4.3org.apache.sparkspark-hive_2.123.4.3org.apache.sparkspark-kubernetes_2.123.4.3org.apache.sparkspark-kvstore_2.123.4.3org.apache.sparkspark-launcher_2.123.4.3org.apache.sparkspark-mesos_2.123.4.3org.apache.sparkspark-mllib-local_2.123.4.3org.apache.sparkspark-mllib_2.123.4.3org.apache.sparkspark-network-common_2.123.4.3org.apache.sparkspark-network-shuffle_2.123.4.3org.apache.sparkspark-repl_2.123.4.3org.apache.sparkspark-sketch_2.123.4.3org.apache.sparkspark-sql_2.123.4.3org.apache.sparkspark-streaming_2.123.4.3org.apache.sparkspark-tags_2.123.4.3org.apache.sparkspark-unsafe_2.123.4.3org.apache.sparkspark-yarn_2.123.4.3org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core3.1.0org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.5org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.3.
The following lists all dependencies provided by this new pipeline environment:
4.0
CHANGED
Updated on 2024-03-21
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.2.
The following lists all dependencies provided by this pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.3.14ch.qos.logbacklogback-core1.3.14com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.15commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.11.0commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.21io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.87.Finalio.nettynetty-buffer4.1.87.Finalio.nettynetty-codec4.1.87.Finalio.nettynetty-codec-http4.1.87.Finalio.nettynetty-codec-http24.1.87.Finalio.nettynetty-codec-socks4.1.87.Finalio.nettynetty-common4.1.87.Finalio.nettynetty-handler4.1.87.Finalio.nettynetty-handler-proxy4.1.87.Finalio.nettynetty-resolver4.1.87.Finalio.nettynetty-transport4.1.87.Finalio.nettynetty-transport-classes-epoll4.1.87.Finalio.nettynetty-transport-classes-kqueue4.1.87.Finalio.nettynetty-transport-native-epoll4.1.87.Finalio.nettynetty-transport-native-epoll4.1.87.Finalio.nettynetty-transport-native-kqueue4.1.87.Finalio.nettynetty-transport-native-kqueue4.1.87.Finalio.nettynetty-transport-native-unix-common4.1.87.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.3org.apache.avroavro-ipc1.11.3org.apache.avroavro-mapred1.11.3org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.22org.apache.commonscommons-configuration22.8.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.12.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.10.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.6org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.2org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.6org.apache.orcorc-mapreduce1.8.6org.apache.orcorc-shims1.8.6org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.13.0org.apache.sparkspark-catalyst_2.123.4.2org.apache.sparkspark-core_2.123.4.2org.apache.sparkspark-graphx_2.123.4.2org.apache.sparkspark-hive-thriftserver_2.123.4.2org.apache.sparkspark-hive_2.123.4.2org.apache.sparkspark-kubernetes_2.123.4.2org.apache.sparkspark-kvstore_2.123.4.2org.apache.sparkspark-launcher_2.123.4.2org.apache.sparkspark-mesos_2.123.4.2org.apache.sparkspark-mllib-local_2.123.4.2org.apache.sparkspark-mllib_2.123.4.2org.apache.sparkspark-network-common_2.123.4.2org.apache.sparkspark-network-shuffle_2.123.4.2org.apache.sparkspark-repl_2.123.4.2org.apache.sparkspark-sketch_2.123.4.2org.apache.sparkspark-sql_2.123.4.2org.apache.sparkspark-streaming_2.123.4.2org.apache.sparkspark-tags_2.123.4.2org.apache.sparkspark-unsafe_2.123.4.2org.apache.sparkspark-yarn_2.123.4.2org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core3.1.0org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.4org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
An existing pipeline environment for batch processing has been changed: batch-4.0 for batch environment with Spark 3.4.2.
The following lists all dependencies provided by this pipeline environment:
4.0
ADDED
Added on 2023-12-19
A new pipeline environment for batch processing has been introduced: batch-4.0 for batch environment with spark 3.4.1.
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionch.qos.logbacklogback-classic1.3.8ch.qos.logbacklogback-core1.3.8com.clearspring.analyticsstream2.9.6com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.14.2com.fasterxml.jackson.corejackson-core2.14.2com.fasterxml.jackson.corejackson-databind2.14.2com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.14.2com.fasterxml.jackson.datatypejackson-datatype-jsr3102.14.2com.fasterxml.jackson.modulejackson-module-scala_2.122.14.2com.fasterxml.woodstoxwoodstox-core5.4.0com.github.joshelserdropwizard-metrics-hadoop-metrics2-reporter0.1.2com.github.lubenzstd-jni1.5.2-5com.github.wendykierpJTransforms3.1com.google.code.findbugsjsr3053.0.0com.google.code.gsongson2.8.9com.google.crypto.tinktink1.7.0com.google.flatbuffersflatbuffers-java1.12.0com.google.guavafailureaccess1.0.1com.google.guavaguava32.1.1-jrecom.google.protobufprotobuf-java2.5.0com.jolboxbonecp0.8.0.RELEASEcom.ningcompress-lzf1.1.2com.squareup.okhttp3logging-interceptor4.12.0com.squareup.okhttp3okhttp4.12.0com.squareup.okiookio3.6.0com.squareup.okiookio-jvm3.6.0com.sun.istackistack-commons-runtime3.0.8com.tdunningjson1.8com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.10.0com.twitterchill_2.120.10.0com.univocityunivocity-parsers2.9.1com.zaxxerHikariCP4.0.3commons-clicommons-cli1.5.0commons-codeccommons-codec1.15commons-collectionscommons-collections3.2.2commons-dbcpcommons-dbcp1.4commons-iocommons-io2.11.0commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-poolcommons-pool1.5.4dev.ludovic.netlibarpack3.0.3dev.ludovic.netlibblas3.0.3dev.ludovic.netliblapack3.0.3io.airliftaircompressor0.21io.dropwizard.metricsmetrics-core4.2.15io.dropwizard.metricsmetrics-graphite4.2.15io.dropwizard.metricsmetrics-jmx4.2.15io.dropwizard.metricsmetrics-json4.2.15io.dropwizard.metricsmetrics-jvm4.2.15io.fabric8kubernetes-client6.4.1io.fabric8kubernetes-client-api6.4.1io.fabric8kubernetes-httpclient-okhttp6.4.1io.fabric8kubernetes-model-admissionregistration6.4.1io.fabric8kubernetes-model-apiextensions6.4.1io.fabric8kubernetes-model-apps6.4.1io.fabric8kubernetes-model-autoscaling6.4.1io.fabric8kubernetes-model-batch6.4.1io.fabric8kubernetes-model-certificates6.4.1io.fabric8kubernetes-model-common6.4.1io.fabric8kubernetes-model-coordination6.4.1io.fabric8kubernetes-model-core6.4.1io.fabric8kubernetes-model-discovery6.4.1io.fabric8kubernetes-model-events6.4.1io.fabric8kubernetes-model-extensions6.4.1io.fabric8kubernetes-model-flowcontrol6.4.1io.fabric8kubernetes-model-gatewayapi6.4.1io.fabric8kubernetes-model-metrics6.4.1io.fabric8kubernetes-model-networking6.4.1io.fabric8kubernetes-model-node6.4.1io.fabric8kubernetes-model-policy6.4.1io.fabric8kubernetes-model-rbac6.4.1io.fabric8kubernetes-model-scheduling6.4.1io.fabric8kubernetes-model-storageclass6.4.1io.fabric8zjsonpatch0.3.0io.nettynetty-all4.1.87.Finalio.nettynetty-buffer4.1.87.Finalio.nettynetty-codec4.1.87.Finalio.nettynetty-codec-http4.1.87.Finalio.nettynetty-codec-http24.1.87.Finalio.nettynetty-codec-socks4.1.87.Finalio.nettynetty-common4.1.87.Finalio.nettynetty-handler4.1.87.Finalio.nettynetty-handler-proxy4.1.87.Finalio.nettynetty-resolver4.1.87.Finalio.nettynetty-transport4.1.87.Finalio.nettynetty-transport-classes-epoll4.1.87.Finalio.nettynetty-transport-classes-kqueue4.1.87.Finalio.nettynetty-transport-native-epoll4.1.87.Finalio.nettynetty-transport-native-epoll4.1.87.Finalio.nettynetty-transport-native-kqueue4.1.87.Finalio.nettynetty-transport-native-kqueue4.1.87.Finalio.nettynetty-transport-native-unix-common4.1.87.Finaljakarta.annotationjakarta.annotation-api1.3.5jakarta.servletjakarta.servlet-api4.0.3jakarta.validationjakarta.validation-api2.0.2jakarta.ws.rsjakarta.ws.rs-api2.1.6jakarta.xml.bindjakarta.xml.bind-api2.3.2javax.activationactivation1.1.1javax.jdojdo-api3.0.1javax.transactionjta1.1javax.transactiontransaction-api1.1javolutionjavolution5.5.1jlinejline2.14.6joda-timejoda-time2.12.2net.openhftzero-allocation-hashing0.9net.razorvinepickle1.3net.sf.jpamjpam1.1net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.9.7net.sf.supercsvsuper-csv2.2.0net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr-runtime3.5.2org.antlrantlr4-runtime4.9.3org.antlrST44.0.4org.apache.arrowarrow-format11.0.0org.apache.arrowarrow-memory-core11.0.0org.apache.arrowarrow-memory-netty11.0.0org.apache.arrowarrow-vector11.0.0org.apache.avroavro1.11.2org.apache.avroavro-ipc1.11.2org.apache.avroavro-mapred1.11.2org.apache.commonscommons-collections44.4org.apache.commonscommons-compress1.22org.apache.commonscommons-configuration22.8.0org.apache.commonscommons-crypto1.1.0org.apache.commonscommons-lang33.12.0org.apache.commonscommons-math33.6.1org.apache.commonscommons-text1.10.0org.apache.curatorcurator-client2.13.0org.apache.curatorcurator-framework2.13.0org.apache.curatorcurator-recipes2.13.0org.apache.derbyderby10.14.2.0org.apache.hadoophadoop-annotations3.3.6org.apache.hadoophadoop-auth3.3.6org.apache.hadoophadoop-aws3.3.6org.apache.hadoophadoop-client3.3.6org.apache.hadoophadoop-client-api3.3.6org.apache.hadoophadoop-client-runtime3.3.6org.apache.hadoophadoop-common3.3.6org.apache.hadoophadoop-hdfs-client3.3.6org.apache.hadoophadoop-mapreduce-client-common3.3.6org.apache.hadoophadoop-mapreduce-client-core3.3.6org.apache.hadoophadoop-mapreduce-client-jobclient3.3.6org.apache.hadoophadoop-yarn-api3.3.6org.apache.hadoophadoop-yarn-client3.3.6org.apache.hadoophadoop-yarn-common3.3.6org.apache.hadoophadoop-yarn-server-web-proxy3.3.4org.apache.hivehive-beeline2.3.9org.apache.hivehive-cli2.3.9org.apache.hivehive-common2.3.9org.apache.hivehive-exec2.3.9org.apache.hivehive-jdbc2.3.9org.apache.hivehive-llap-common2.3.9org.apache.hivehive-metastore2.3.9org.apache.hivehive-serde2.3.9org.apache.hivehive-service-rpc3.1.3org.apache.hivehive-shims2.3.9org.apache.hivehive-storage-api2.8.1org.apache.hive.shimshive-shims-0.232.3.9org.apache.hive.shimshive-shims-common2.3.9org.apache.hive.shimshive-shims-scheduler2.3.9org.apache.httpcomponentshttpclient4.5.14org.apache.httpcomponentshttpcore4.4.16org.apache.ivyivy2.5.1org.apache.logging.log4jlog4j-api2.19.0org.apache.logging.log4jlog4j-core2.19.0org.apache.logging.log4jlog4j-to-slf4j2.19.0org.apache.mesosmesos1.11.0org.apache.orcorc-core1.8.4org.apache.orcorc-mapreduce1.8.4org.apache.orcorc-shims1.8.4org.apache.parquetparquet-column1.12.3org.apache.parquetparquet-common1.12.3org.apache.parquetparquet-encoding1.12.3org.apache.parquetparquet-format-structures1.12.3org.apache.parquetparquet-hadoop1.12.3org.apache.parquetparquet-jackson1.12.3org.apache.sparkspark-catalyst_2.123.4.1org.apache.sparkspark-core_2.123.4.1org.apache.sparkspark-graphx_2.123.4.1org.apache.sparkspark-hive-thriftserver_2.123.4.1org.apache.sparkspark-hive_2.123.4.1org.apache.sparkspark-kubernetes_2.123.4.1org.apache.sparkspark-kvstore_2.123.4.1org.apache.sparkspark-launcher_2.123.4.1org.apache.sparkspark-mesos_2.123.4.1org.apache.sparkspark-mllib-local_2.123.4.1org.apache.sparkspark-mllib_2.123.4.1org.apache.sparkspark-network-common_2.123.4.1org.apache.sparkspark-network-shuffle_2.123.4.1org.apache.sparkspark-repl_2.123.4.1org.apache.sparkspark-sketch_2.123.4.1org.apache.sparkspark-sql_2.123.4.1org.apache.sparkspark-streaming_2.123.4.1org.apache.sparkspark-tags_2.123.4.1org.apache.sparkspark-unsafe_2.123.4.1org.apache.sparkspark-yarn_2.123.4.1org.apache.thriftlibfb3030.9.3org.apache.thriftlibthrift0.14.1org.apache.xbeanxbean-asm9-shaded4.22org.apache.yetusaudience-annotations0.5.0org.apache.zookeeperzookeeper3.6.3org.apache.zookeeperzookeeper-jute3.6.3org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.janinocommons-compiler3.1.9org.codehaus.janinojanino3.1.9org.codehaus.woodstoxstax2-api4.2.1org.datanucleusdatanucleus-api-jdo4.2.4org.datanucleusdatanucleus-core4.1.17org.datanucleusdatanucleus-rdbms4.1.19org.datanucleusjavax.jdo3.2.0-m3org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.6.1org.glassfish.hk2hk2-locator2.6.1org.glassfish.hk2hk2-utils2.6.1org.glassfish.hk2osgi-resource-locator1.0.3org.glassfish.hk2.externalaopalliance-repackaged2.6.1org.glassfish.hk2.externaljakarta.inject2.6.1org.glassfish.jaxbjaxb-runtime2.3.2org.glassfish.jersey.containersjersey-container-servlet2.36org.glassfish.jersey.containersjersey-container-servlet-core2.36org.glassfish.jersey.corejersey-client2.36org.glassfish.jersey.corejersey-common2.36org.glassfish.jersey.corejersey-server2.36org.glassfish.jersey.injectjersey-hk22.36org.javassistjavassist3.25.0-GAorg.jctoolsjctools-core3.1.0org.jetbrainsannotations17.0.0org.jetbrains.kotlinkotlin-stdlib1.9.10org.jetbrains.kotlinkotlin-stdlib-common1.9.10org.jetbrains.kotlinkotlin-stdlib-jdk71.9.10org.jetbrains.kotlinkotlin-stdlib-jdk81.9.10org.joddjodd-core5.0.13org.json4sjson4s-ast_2.123.7.0-M11org.json4sjson4s-core_2.123.7.0-M11org.json4sjson4s-jackson_2.123.7.0-M11org.json4sjson4s-scalap_2.123.7.0-M11org.lz4lz4-java1.8.0org.objenesisobjenesis3.2org.roaringbitmapRoaringBitmap0.9.38org.roaringbitmapshims0.9.38org.rocksdbrocksdbjni7.9.2org.scala-langscala-library2.12.17org.scala-langscala-reflect2.12.17org.scala-lang.modulesscala-collection-compat_2.122.7.0org.scala-lang.modulesscala-parser-combinators_2.122.1.1org.scala-lang.modulesscala-xml_2.122.1.0org.scalanlpbreeze-macros_2.122.1.0org.scalanlpbreeze_2.122.1.0org.slf4jjcl-over-slf4j2.0.6org.slf4jjul-to-slf4j2.0.6org.slf4jlog4j-over-slf4j2.0.7org.slf4jslf4j-api2.0.6org.threetenthreeten-extra1.7.1org.tukaanixz1.9org.typelevelalgebra_2.122.0.1org.typelevelcats-kernel_2.122.6.1org.typelevelspire-macros_2.120.17.0org.typelevelspire-platform_2.120.17.0org.typelevelspire-util_2.120.17.0org.typelevelspire_2.120.17.0org.wildfly.opensslwildfly-openssl1.1.3.Finalorg.wildfly.opensslwildfly-openssl-java1.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-linux-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-macosx-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-solaris-x86_641.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-i3861.1.3.Finalorg.wildfly.opensslwildfly-openssl-windows-x86_641.1.3.Finalorg.xerial.snappysnappy-java1.1.10.1org.yamlsnakeyaml2.0orooro2.0.8pl.edu.icmJLargeArrays1.5staxstax-api1.0.1
A new pipeline environment for batch processing has been introduced: batch-4.0 for batch environment with spark 3.4.1.
The following lists all dependencies provided by this new pipeline environment:
3.0
DEPRECATED
Deprecated on 2023-12-19
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 6 months:
batch-3.0 for batch environment with Spark 2.4.7
Please, use the latest SDK to migrate your applications to this new environment:
batch-4.0 for batch environment with Spark 3.4.1
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 6 months:
batch-3.0 for batch environment with Spark 2.4.7
Please, use the latest SDK to migrate your applications to this new environment:
batch-4.0 for batch environment with Spark 3.4.1
2.1.0
DEPRECATED
Deprecated on 2021-06-23
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 6 months:
batch-2.1.0 for batch environment with Spark 2.4.2
Please, use the latest SDK to migrate your applications to Scala 2.12 and to this new environment:
batch-3.0 for batch environment with Spark 2.4.7
With the introduction of a new pipeline environment for batch processing the following environment has been deprecated and will be removed after 6 months:
batch-2.1.0 for batch environment with Spark 2.4.2
Please, use the latest SDK to migrate your applications to Scala 2.12 and to this new environment:
batch-3.0 for batch environment with Spark 2.4.7
2.1.0
ADDED
Added on 2019-12-18
A new pipeline environments for batch processing has been introduced to update the Spark version:
batch-2.1.0 for batch environment with Spark 2.4.2
The following lists all dependencies provided by this new pipeline environment:
Group IDArtifact IDVersionaopallianceaopalliance1.0ch.qos.logbacklogback-classic1.2.3ch.qos.logbacklogback-core1.2.3com.amazonawsaws-java-sdk1.7.4com.carrotsearchhppc0.7.2com.chuusaishapeless_2.112.3.2com.clearspring.analyticsstream2.7.0com.esotericsoftwarekryo-shaded4.0.2com.esotericsoftwareminlog1.3.0com.esotericsoftwarereflectasm1.11.3com.fasterxml.jackson.corejackson-annotations2.6.7com.fasterxml.jackson.corejackson-core2.6.7com.fasterxml.jackson.corejackson-databind2.6.7.1com.fasterxml.jackson.dataformatjackson-dataformat-yaml2.6.7com.fasterxml.jackson.modulejackson-module-jaxb-annotations2.6.7com.fasterxml.jackson.modulejackson-module-paranamer2.7.9com.fasterxml.jackson.modulejackson-module-scala_2.112.6.7.1com.github.fommil.netlibcore1.1.2com.github.lubenzstd-jni1.3.2-2com.github.mifmifgenerex1.0.1com.github.rwljtransforms2.4.0com.google.code.findbugsjsr3051.3.9com.google.code.gsongson2.2.4com.google.guavaguava14.0.1com.google.injectguice3.0com.google.inject.extensionsguice-servlet3.0com.google.protobufprotobuf-java2.5.0com.ningcompress-lzf1.0.3com.squareup.okhttp3logging-interceptor3.12.0com.squareup.okhttp3okhttp3.12.10com.squareup.okiookio1.15.0com.thoughtworks.paranamerparanamer2.8com.twitterchill-java0.9.3com.twitterchill_2.110.9.3com.univocityunivocity-parsers2.7.3com.vlkanflatbuffers1.2.0-3f79e055commons-beanutilscommons-beanutils1.7.0commons-beanutilscommons-beanutils-core1.8.0commons-clicommons-cli1.2commons-codeccommons-codec1.10commons-collectionscommons-collections3.2.2commons-configurationcommons-configuration1.6commons-digestercommons-digester1.8commons-httpclientcommons-httpclient3.1commons-iocommons-io2.4commons-langcommons-lang2.6commons-loggingcommons-logging1.1.3commons-netcommons-net3.1dk.brics.automatonautomaton1.11-8io.airliftaircompressor0.10io.fabric8kubernetes-client4.1.2io.fabric8kubernetes-model4.1.2io.fabric8kubernetes-model-common4.1.2io.fabric8zjsonpatch0.3.0io.nettynetty3.9.9.Finalio.nettynetty-all4.1.17.Finaljavax.activationactivation1.1.1javax.annotationjavax.annotation-api1.2javax.injectjavax.inject1javax.servletjavax.servlet-api3.1.0javax.servlet.jspjsp-api2.1javax.validationvalidation-api1.1.0.Finaljavax.ws.rsjavax.ws.rs-api2.0.1javax.xml.bindjaxb-api2.2.2javax.xml.streamstax-api1.0-2joda-timejoda-time2.9.3log4jlog4j1.2.17net.razorvinepyrolite4.13net.sf.opencsvopencsv2.3net.sf.py4jpy4j0.10.7net.sourceforge.f2jarpack_combined_all0.1org.antlrantlr4-runtime4.7org.apache.arrowarrow-format0.10.0org.apache.arrowarrow-memory0.10.0org.apache.arrowarrow-vector0.10.0org.apache.avroavro1.8.2org.apache.avroavro-guava-dependencies1.8.2org.apache.avroavro-mapred1.8.2org.apache.commonscommons-compress1.8.1org.apache.commonscommons-crypto1.0.0org.apache.commonscommons-lang33.5org.apache.commonscommons-math33.4.1org.apache.curatorcurator-client2.7.1org.apache.curatorcurator-framework2.7.1org.apache.curatorcurator-recipes2.7.1org.apache.directory.apiapi-asn1-api1.0.0-M20org.apache.directory.apiapi-util1.0.0-M20org.apache.directory.serverapacheds-i18n2.0.0-M15org.apache.directory.serverapacheds-kerberos-codec2.0.0-M15org.apache.hadoophadoop-annotations2.7.3org.apache.hadoophadoop-auth2.7.3org.apache.hadoophadoop-aws2.7.3org.apache.hadoophadoop-client2.7.3org.apache.hadoophadoop-common2.7.3org.apache.hadoophadoop-hdfs2.7.3org.apache.hadoophadoop-mapreduce-client-app2.7.3org.apache.hadoophadoop-mapreduce-client-common2.7.3org.apache.hadoophadoop-mapreduce-client-core2.7.3org.apache.hadoophadoop-mapreduce-client-jobclient2.7.3org.apache.hadoophadoop-mapreduce-client-shuffle2.7.3org.apache.hadoophadoop-yarn-api2.7.3org.apache.hadoophadoop-yarn-client2.7.3org.apache.hadoophadoop-yarn-common2.7.3org.apache.hadoophadoop-yarn-server-common2.7.3org.apache.htracehtrace-core3.1.0-incubatingorg.apache.httpcomponentshttpclient4.5.6org.apache.httpcomponentshttpcore4.4.10org.apache.ivyivy2.4.0org.apache.mesosmesos1.4.0org.apache.orcorc-core1.5.5org.apache.orcorc-mapreduce1.5.5org.apache.orcorc-shims1.5.5org.apache.parquetparquet-column1.10.1org.apache.parquetparquet-common1.10.1org.apache.parquetparquet-encoding1.10.1org.apache.parquetparquet-format2.4.0org.apache.parquetparquet-hadoop1.10.1org.apache.parquetparquet-jackson1.10.1org.apache.sparkspark-catalyst_2.112.4.2org.apache.sparkspark-core_2.112.4.2org.apache.sparkspark-graphx_2.112.4.2org.apache.sparkspark-kubernetes_2.112.4.2org.apache.sparkspark-kvstore_2.112.4.2org.apache.sparkspark-launcher_2.112.4.2org.apache.sparkspark-mesos_2.112.4.2org.apache.sparkspark-mllib-local_2.112.4.2org.apache.sparkspark-mllib_2.112.4.2org.apache.sparkspark-network-common_2.112.4.2org.apache.sparkspark-network-shuffle_2.112.4.2org.apache.sparkspark-repl_2.112.4.2org.apache.sparkspark-sketch_2.112.4.2org.apache.sparkspark-sql_2.112.4.2org.apache.sparkspark-streaming_2.112.4.2org.apache.sparkspark-tags_2.112.4.2org.apache.sparkspark-unsafe_2.112.4.2org.apache.sparkspark-yarn_2.112.4.2org.apache.xbeanxbean-asm6-shaded4.8org.apache.zookeeperzookeeper3.4.6org.codehaus.jacksonjackson-core-asl1.9.13org.codehaus.jacksonjackson-jaxrs1.9.13org.codehaus.jacksonjackson-mapper-asl1.9.13org.codehaus.jacksonjackson-xc1.9.13org.codehaus.janinocommons-compiler3.0.9org.codehaus.janinojanino3.0.9org.fusesource.leveldbjnileveldbjni-all1.8org.glassfish.hk2hk2-api2.4.0-b34org.glassfish.hk2hk2-locator2.4.0-b34org.glassfish.hk2hk2-utils2.4.0-b34org.glassfish.hk2osgi-resource-locator1.0.1org.glassfish.hk2.externalaopalliance-repackaged2.4.0-b34org.glassfish.hk2.externaljavax.inject2.4.0-b34org.glassfish.jersey.bundles.repackagedjersey-guava2.22.2org.glassfish.jersey.containersjersey-container-servlet2.22.2org.glassfish.jersey.containersjersey-container-servlet-core2.22.2org.glassfish.jersey.corejersey-client2.22.2org.glassfish.jersey.corejersey-common2.22.2org.glassfish.jersey.corejersey-server2.22.2org.glassfish.jersey.mediajersey-media-jaxb2.22.2org.javassistjavassist3.18.1-GAorg.json4sjson4s-ast_2.113.5.3org.json4sjson4s-core_2.113.5.3org.json4sjson4s-jackson_2.113.5.3org.json4sjson4s-scalap_2.113.5.3org.jvnettiger-types1.4org.lz4lz4-java1.4.0org.mortbay.jettyjetty6.1.26org.mortbay.jettyjetty-util6.1.26org.objenesisobjenesis2.5.1org.roaringbitmapRoaringBitmap0.7.45org.roaringbitmapshims0.7.45org.scala-langscala-compiler2.11.12org.scala-langscala-library2.11.12org.scala-langscala-reflect2.11.12org.scala-lang.modulesscala-parser-combinators_2.111.1.0org.scala-lang.modulesscala-xml_2.111.0.5org.scalanlpbreeze-macros_2.110.13.2org.scalanlpbreeze_2.110.13.2org.slf4jjcl-over-slf4j1.7.16org.slf4jjul-to-slf4j1.7.16org.slf4jlog4j-over-slf4j1.7.16org.slf4jslf4j-api1.7.16org.spire-mathspire-macros_2.110.13.0org.spire-mathspire_2.110.13.0org.tukaanixz1.5org.typelevelmachinist_2.110.6.1org.typelevelmacro-compat_2.111.1.1org.xerial.snappysnappy-java1.1.7.3org.yamlsnakeyaml1.15orooro2.0.8xercesxercesImpl2.9.1xmlencxmlenc0.52
A new pipeline environments for batch processing has been introduced to update the Spark version:
batch-2.1.0 for batch environment with Spark 2.4.2
The following lists all dependencies provided by this new pipeline environment: