Header Shadow Image


Archive for the 'NIX Posts' Category

WARN lineage.LineageWriter: Lineage directory /var/log/spark/lineage doesn’t exist or is not writable. Lineage for this application will be disabled.

Gettnig this? 19/09/17 22:17:41 WARN lineage.LineageWriter: Lineage directory /var/log/spark/lineage doesn't exist or is not writable. Lineage for this application will be disabled. Resolve it by creating the folder: [root@cm-r01en01 spark]# ls -altri total 2244 335565630 drwxr-xr-x.  2 spark spark       6 Aug 17 21:25 stacks  67109037 drwxr-xr-x. 27 root  root     4096 […]

FATAL: remaining connection slots are reserved for non-replication superuser connections

Getting this? FATAL:  remaining connection slots are reserved for non-replication superuser connections Fix that by updating the Patroni configuration like like so: [root@psql01 log]# patronictl -c /etc/patroni.yml edit-config postgres — +++ @@ -1,9 +1,10 @@  loop_wait: 10  maximum_lag_on_failover: 1048576  postgresql: +  parameters: –  max_connections: 256 +    max_connections: 256 –  max_replication_slots: 64 +    max_replication_slots: […]

touch: cannot touch /atlas/atlassian/confluence/logs/catalina.out: Permission denied

Getting this? [confluence@atlas02 logs]$ logout [root@atlas02 atlassian]# systemctl status confluence.service -l â confluence.service – LSB: Atlassian Confluence    Loaded: loaded (/etc/rc.d/init.d/confluence; bad; vendor preset: disabled)    Active: failed (Result: exit-code) since Tue 2019-09-10 22:07:18 EDT; 2min 5s ago      Docs: man:systemd-sysv-generator(8)   Process: 11361 ExecStop=/etc/rc.d/init.d/confluence stop (code=exited, status=0/SUCCESS)   Process: 11925 ExecStart=/etc/rc.d/init.d/confluence start (code=exited, […]

Application application_1567571625367_0006 failed 2 times due to AM Container for appattempt_1567571625367_0006_000002 exited with  exitCode: -1000

Getting this? 19/09/07 23:41:56 ERROR repl.Main: Failed to initialize Spark session. org.apache.spark.SparkException: Application application_1567571625367_0006 failed 2 times due to AM Container for appattempt_1567571625367_0006_000002 exited with  exitCode: -1000 Failing this attempt.Diagnostics: [2019-09-07 23:41:54.934]Application application_1567571625367_0006 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is tom main : requested yarn user […]

Configure Cloudera HUE with FreeIPA

Configuring HUE with LDAP / FreeIPA: [root@idmipa03 ~]# LDAPTLS_CACERT=/etc/ipa/ca.crt ldapsearch -H ldaps://idmipa03.mws.mds.xyz:636 -D "uid=admin,cn=users,cn=compat,dc=mws,dc=mds,dc=xyz" -w "<SECRET>" -b "dc=mws,dc=mds,dc=xyz" -v "(&(objectClass=posixAccount)(uid=*))"  |grep dn: ldap_initialize( ldaps://idmipa03.mws.mds.xyz:636/??base ) filter: (&(objectClass=posixAccount)(uid=*)) requesting: All userApplication attributes dn: uid=cmadmin-530029b6,cn=users,cn=compat,dc=mws,dc=mds,dc=xyz dn: uid=admin,cn=users,cn=compat,dc=mws,dc=mds,dc=xyz dn: uid=admin,cn=users,cn=accounts,dc=mws,dc=mds,dc=xyz dn: uid=cmadmin-530029b6,cn=users,cn=accounts,dc=mws,dc=mds,dc=xyz [root@idmipa03 ~]# Ensure the following settings: Authentication Backend ( backend ) : desktop.authentication.backend.LdapBackend PAM Backend Service […]

There is a problem processing audits for HIVESERVER2.

Getting this? There is a problem processing audits for HIVESERVER2. [02/Sep/2019 12:36:30 +0000] 32165 Audit-Plugin throttling_logger ERROR    (341 skipped) Error occurred when sending entry to server: [02/Sep/2019 12:36:30 +0000] 32165 Audit-Plugin throttling_logger INFO     (341 skipped) Unable to send data to nav server. Will try again. Diggig further, we see this error as […]

How do I connect to HiveServer2 (HS2) through beeline

How do I connect to HiveServer2 (HS2) through beeline: beeline> !connect jdbc:hive2://cm-r01en01.mws.mds.xyz:10000/default;principal=hive/cm-r01en01.mws.mds.xyz@MWS.MDS.XYZ Connecting to jdbc:hive2://cm-r01en01.mws.mds.xyz:10000/default;principal=hive/cm-r01en01.mws.mds.xyz@MWS.MDS.XYZ Connected to: Apache Hive (version 2.1.1-cdh6.3.0) Driver: Hive JDBC (version 2.1.1-cdh6.3.0) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://cm-r01en01.mws.mds.xyz:10000/> 0: jdbc:hive2://cm-r01en01.mws.mds.xyz:10000/> 0: jdbc:hive2://cm-r01en01.mws.mds.xyz:10000/> show tables; INFO  : Compiling command(queryId=hive_20190902102937_4ef97c5b-19ff-47b8-be81-dacb2edeece0): show tables INFO  : Semantic Analysis Completed INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, […]

Required executor memory (1024), overhead (384 MB), and PySpark memory (0 MB) is above the max threshold (1024 MB) of this cluster!

Getting this? java.lang.IllegalArgumentException: Required executor memory (1024), overhead (384 MB), and PySpark memory (0 MB) is above the max threshold (1024 MB) of this cluster! Please check the values of 'yarn.scheduler.maximum-allocation-mb' and/or 'yarn.nodemanager.resource.memory-mb'. Either increase the Container Memory: yarn.nodemanager.resource.memory-mb to 2GB from 1GB Or reduce the maximum container memory: yarn.scheduler.maximum-allocation-mb from 1GB to 512MB in the […]

Cloudera Clusters: Running kinit on a kerberos principal returns with a 1969 date.

Are you getting this result? [root@cm-r01en01 process]# systemctl restart cloudera-scm-agent [root@cm-r01en01 process]# [root@cm-r01en01 process]# [root@cm-r01en01 process]# [root@cm-r01en01 process]# kinit -kt ./1401-hdfs-NFSGATEWAY/hdfs.keytab hdfs/cm-r01en01.mws.mds.xyz@MWS.MDS.XYZ [root@cm-r01en01 process]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: host/cm-r01en01.mws.mds.xyz@MWS.MDS.XYZ Valid starting       Expires              Service principal 12/31/1969 19:00:00  12/31/1969 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY: [root@cm-r01en01 process]#   Solve it […]

org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]

Getting this? [root@cm-r01en01 ~]# hdfs dfs -ls / 19/08/25 22:43:19 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] ls: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "cm-r01en01.mws.mds.xyz/192.168.0.140"; destination host is: "cm-r01nn02.mws.mds.xyz":8020; [root@cm-r01en01 ~]# Fix it by doing […]


     
  Copyright © 2003 - 2013 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License