Story 

I ran log parser within docker containers to parse log file that mount from the host, and everything goes well until daily log rotation. Container seems failed to mount the correct file foo.log but foo.log.1.

Try It Yourself 

You can try it with following steps

• docker-compose.yml

dummy-log-parser:
  image: alpine:latest
  volumes:
    - /tmp/foo.log:/tmp/foo.log:ro
  command:
    - "sleep"
    - "3600"

• execute following commands

$ touch /tmp/foo.log
$ docker-compose up -d
$ echo 'foo' >> /tmp/foo.log

• check file content in container, and you should see foo in foo.log

$ docker exec -it <container> sh 
$ cat /tmp/foo.log
foo

• the tricky part

# on host
$ mv /tmp/foo.log /tmp/foo.log.1
$ touch /tmp/foo.log
$ echo 'foo bar foo bar' >> /tmp/foo.log
$ cat /tmp/foo.log
foo bar foo bar
$ echo 'old log file' >> /tmp/foo.log.1
$ cat /tmp/foo.log.1
foo
old log file

# in container
$ cat /tmp/foo.log
foo
old log file

WHAT?

What happened 

The filesystem has an inode table that stores the mapping of file and inode number. When container ups, the log file is mounted into the container with specific inode number. Even the file name is changed on host, the inode number remains the same. That’s why the file in container still points to the old log file.

Solution 

Mount directory rather than mounting file!

dummy-log-parser:
  image: alpine:latest
  volumes:
    - /tmp/:/tmp/:ro
  command:
    - "sleep"
    - "3600"

Reference 

• https://github.com/docker/docker/issues/15793#issuecomment-135411504
• https://forums.docker.com/t/modify-a-file-which-mount-as-a-data-volume-but-it-didnt-change-in-container/2813/14

See Also

• Kubernetes - Two Steps Installation
• Adopting Container and Kubernetes in Production
• Building Minimal Docker Image for Go Applications
• Rolling Updates with Kubernetes Deployments
• Kubernetes - Assigning Pod to Nodes