AttributeError: ‘SparkContext’ object has no attribute ‘list_packages’ – AWS EMR
I am trying to install a new package in AWS EMR and running into the error “AttributeError: ‘SparkContext’ object has no attribute ‘list_packages'”. However, this seems to be the official approach. Appreciate your help in finding what I am missing
Code
from pyspark.sql import SparkSession, Window
from pyspark import SparkConf, SparkContext
spark = SparkSession.builder.getOrCreate()
sc =SparkContext.getOrCreate()
sc.list_packages()
Config emr-5.20.0 Spark 2.4.0