Getting this?

19/09/07 23:41:56 ERROR repl.Main: Failed to initialize Spark session.
org.apache.spark.SparkException: Application application_1567571625367_0006 failed 2 times due to AM Container for appattempt_1567571625367_0006_000002 exited with  exitCode: -1000
Failing this attempt.Diagnostics: [2019-09-07 23:41:54.934]Application application_1567571625367_0006 initialization failed (exitCode=255) with output: main : command provided 0
main : run as user is tom
main : requested yarn user is tom
User tom not found

For more detailed output, check the application tracking page: http://cm-r01nn02.mws.mds.xyz:8088/cluster/app/application_1567571625367_0006 Then click on links to logs of each attempt.
. Failing the application.
 

This is likely due to incorrect auth_to_local rules in HDFS -> Configuration:

RULE:[2:$1@$0](HTTP@\QMWS.MDS.XYZ\E$)s/@\QMWS.MDS.XYZ\E$//
RULE:[1:$1@$0](.*@\QMWS.MDS.XYZ\E$)s/@\QMWS.MDS.XYZ\E$///L
RULE:[2:$1@$0](.*@\QMWS.MDS.XYZ\E$)s/@\QMWS.MDS.XYZ\E$///L
RULE:[2:$1@$0](HTTP@\Qmws.mds.xyz\E$)s/@\Qmws.mds.xyz\E$//
RULE:[1:$1@$0](.*@\Qmws.mds.xyz\E$)s/@\Qmws.mds.xyz\E$///L
RULE:[2:$1@$0](.*@\Qmws.mds.xyz\E$)s/@\Qmws.mds.xyz\E$///L
RULE:[2:$1@$0](HTTP@\QMDS.XYZ\E$)s/@\QMDS.XYZ\E$//
RULE:[1:$1@$0](.*@\QMDS.XYZ\E$)s/@\QMDS.XYZ\E$///L
RULE:[2:$1@$0](.*@\QMDS.XYZ\E$)s/@\QMDS.XYZ\E$///L
RULE:[2:$1@$0](HTTP@\Qmds.xyz\E$)s/@\Qmds.xyz\E$//
RULE:[1:$1@$0](.*@\Qmds.xyz\E$)s/@\Qmds.xyz\E$///L
RULE:[2:$1@$0](.*@\Qmds.xyz\E$)s/@\Qmds.xyz\E$///L
 

In our case, removed the above rules.  More fine-tuning would be needed to make them both HDFS and Spark friendly. 

Cheers,
TK