GCLOUD_DATAPROC_JOBS_SUBMIT(1)

NAME

gcloud dataproc jobs submit - submit Google Cloud Dataproc jobs to execute on a cluster

SYNOPSIS

gcloud dataproc jobs submit COMMAND [--async] [--bucket BUCKET] [GLOBAL-FLAG ...]

DESCRIPTION
Submit Google Cloud Dataproc jobs to execute on a cluster.

FLAGS
--async
    Does not wait for the job to run.
--bucket BUCKET
    The Cloud Storage bucket to stage files in. Defaults to the cluster's configured bucket.

GLOBAL FLAGS
Run $ gcloud help for a description of flags available to all commands.

COMMANDS
COMMAND is one of the following: hadoop
    Submit a Hadoop job to a cluster.
hive
    Submit a Hive job to a cluster.
pig
    Submit a Pig job to a cluster.
pyspark
    Submit a PySpark job to a cluster.
spark
    Submit a Java or Scala Spark job to a cluster.
spark-sql
    Submit a Spark SQL job to a cluster.

EXAMPLES
To submit a Hadoop MapReduce job, run:
    $ gcloud dataproc jobs submit hadoop --cluster my_cluster \
        --jar my_jar.jar arg1 arg2
To submit a Spark Scala or Java job, run:
    $ gcloud dataproc jobs submit spark --cluster my_cluster \
        --jar my_jar.jar arg1 arg2
To submit a PySpark job, run:
    $ gcloud dataproc jobs submit pyspark --cluster my_cluster \
        my_script.py arg1 arg2
To submit a Spark SQL job, run:
    $ gcloud dataproc jobs submit spark-sql --cluster my_cluster \
        --file my_queries.q
To submit a Pig job, run:
    $ gcloud dataproc jobs submit pig --cluster my_cluster \
        --file my_script.pig
To submit a Hive job, run:
    $ gcloud dataproc jobs submit hive --cluster my_cluster \
        --file my_queries.q