Friday, February 12, 2016

Too many open files

If you are working with application runs on Linux based OS that involves in lots of I/O operations you may have encountered this error."Too many open files (24)"

What is This error
In Linux based Os's there are resource limits are specified for user/process to ensure fair usage of resources and for security reasons.If a resource usage of  of user/process try exceed the specified limit it was prevented by the OS.

How to see this limits
By using ulimit command we can examine this parameters at the global level.

[user@localhost ~]$ ulimit -a
core file size           (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7281
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
to see this details in a more specified way we can use /proc folder with relevant process id.
[user@localhost ~]$ sudo cat /proc/989/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             7281                 7281                 processes
Max open files            1024                 4096                 files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       7281                 7281                 signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us
to see file descriptor limit currently used by the process

[user@localhost ~]$ sudo ls -al /proc/1181/fd | wc -l
17

Using lsof command sometimes not provide accurate detail because it count all the files involves with the process even .so files.

How to increase this limits 

In the OS level

Enter value like the following to the /etc/sysctl.conf.(Maximum number can be allowed is 65535 because it is the highest number which can be represented by an unsigned 16-bit binary number) and reload the kernel variables.
[user@localhost ~]$ cat /etc/sysctl.conf
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
fs.file-max=20000
[user@localhost ~]$ sudo sysctl -p
[sudo] password for user:
[user@localhost ~]$ sysctl fs.file-max
fs.file-max = 20000
In the user level enter the new configuration to /etc/security/limits.conf file using following format.
user       soft    nofile   10000
user       hard    nofile  8000


Thursday, February 11, 2016

Java trust store and key store

To ensure secure communication over internet we can use Secure Sockets Layer (SSL) which uses Public-key cryptography. Public-key cryptography is based on the concept of a key pair, which consists of a two keys. Data that has been encrypted with a one key can be decrypted only with the corresponding other key.

Keytool Program
The keytool program is a security tool included in the bin directory of the Java SDK. Which can be used to manage public key cryptography key pairs. It can used to manage key databases. Mainly there are two type of databases.

KeyStore 
Which is used when a java program act as a server. That contains both private and corresponding key pairs.

TrustStore
Which contains public keys which is used when it act as client. When a server present it’s key to the client it check can I trust this certificate. To do so it checks is it signed by a Trusted Certificate Authority (which public key of the CA is with its trust store).

Creating a trust store with CA certificates
[user@localhost certificates]$ ls -al
total 8
-rw-rw-r--. 1 user user 1360 Feb 11  2016 DigiCertGlobalRootCA.crt
-rw-rw-r--. 1 user user  806 Apr 29  2009 ThawteServerCA.cer
[user@localhost certificates]$ keytool  -import -file DigiCertGlobalRootCA.crt -alias DigiCertGlobalRootCA -keystore TrustStore
Enter keystore password:
Re-enter new password:
Owner: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Issuer: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: 83be056904246b1a1756ac95991c74a
Valid from: Fri Nov 10 05:30:00 IST 2006 until: Mon Nov 10 05:30:00 IST 2031
Certificate fingerprints:
         MD5:  79:E4:A9:84:0D:7D:3A:96:D7:C0:4F:E2:43:4C:89:2E
         SHA1: A8:98:5D:3A:65:E5:E5:C4:B2:D7:D6:6D:40:C6:DD:2F:B1:9C:54:36
         SHA256: 43:48:A0:E9:44:4C:78:CB:26:5E:05:8D:5E:89:44:B4:D8:4F:96:62:BD:26:DB:25:7F:89:34:A4:43:C7:01:61
         Signature algorithm name: SHA1withRSA
         Version: 3

Extensions:

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 03 DE 50 35 56 D1 4C BB   66 F0 A3 E2 1B 1B C3 97  ..P5V.L.f.......
0010: B2 3D D1 55                                        .=.U
]
]

#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_CertSign
  Crl_Sign
]

#4: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 03 DE 50 35 56 D1 4C BB   66 F0 A3 E2 1B 1B C3 97  ..P5V.L.f.......
0010: B2 3D D1 55                                        .=.U
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
[user@localhost certificates]$ keytool  -import -file ThawteServerCA.cer -alias ThawteServerCA -keystore TrustStore
Enter keystore password:
Owner: EMAILADDRESS=server-certs@thawte.com, CN=Thawte Server CA, OU=Certification Services Division, O=Thawte Consulting cc, L=Cape Town, ST=Western Cape, C=ZA
Issuer: EMAILADDRESS=server-certs@thawte.com, CN=Thawte Server CA, OU=Certification Services Division, O=Thawte Consulting cc, L=Cape Town, ST=Western Cape, C=ZA
Serial number: 34a4fff630af4ca53c331742a1946675
Valid from: Thu Aug 01 06:30:00 IST 1996 until: Sat Jan 02 05:29:59 IST 2021
Certificate fingerprints:
         MD5:  EE:FE:61:69:65:6E:F8:9C:C6:2A:F4:D7:2B:63:EF:A2
         SHA1: 9F:AD:91:A6:CE:6A:C6:C5:00:47:C4:4E:C9:D4:A5:0D:92:D8:49:79
         SHA256: 87:C6:78:BF:B8:B2:5F:38:F7:E9:7B:33:69:56:BB:CF:14:4B:BA:CA:A5:36:47:E6:1A:23:25:BC:10:55:31:6B
         Signature algorithm name: SHA1withRSA
         Version: 3

Extensions:

#1: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
[user@localhost certificates]$ keytool  -list -keystore TrustStore
Enter keystore password:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 2 entries

digicertglobalrootca, Feb 11, 2016, trustedCertEntry,
Certificate fingerprint (SHA1): A8:98:5D:3A:65:E5:E5:C4:B2:D7:D6:6D:40:C6:DD:2F:B1:9C:54:36
thawteserverca, Feb 11, 2016, trustedCertEntry,
Certificate fingerprint (SHA1): 9F:AD:91:A6:CE:6A:C6:C5:00:47:C4:4E:C9:D4:A5:0D:92:D8:49:79
[user@localhost certificates]$