'builds' directory in a separate location?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

'builds' directory in a separate location?

Michael March
Is there any way in Hudson to have the 'builds' directories not be a branch off the "jobs/job_name" hierarchy? 

thanks!
--
<admiral>

Reply | Threaded
Open this post in threaded view
|

Re: 'builds' directory in a separate location?

Francois Cottet
Yes have a similar concern: can we separate the storage of previous builds from the workspace?

The idea would be to store the previous builds on a large NAS and to execute the jobs on a fast disk (a SSD drive for example).

Thanks for your suggestion,
Francois


Michael March wrote
Is there any way in Hudson to have the 'builds' directories not be a branch
off the "jobs/job_name" hierarchy?
Reply | Threaded
Open this post in threaded view
|

Re: 'builds' directory in a separate location?

Michael March
Hehe.. that is EXACTLY what I am trying to do.

On Thu, Oct 14, 2010 at 12:07 AM, Francois Cottet <[hidden email]> wrote:

Yes have a similar concern: can we separate the storage of previous builds
from the workspace?

The idea would be to store the previous builds on a large NAS and to execute
the jobs on a fast disk (a SSD drive for example).

Thanks for your suggestion,
Francois


Reply | Threaded
Open this post in threaded view
|

Re: 'builds' directory in a separate location?

Michael March
See below...


> Hehe.. that is EXACTLY what I am trying to do.
>
> On Thu, Oct 14, 2010 at 12:07 AM, Francois Cottet <[hidden email]> wrote:
>>
>> Yes have a similar concern: can we separate the storage of previous builds
>> from the workspace?
>>
>> The idea would be to store the previous builds on a large NAS and to execute
>> the jobs on a fast disk (a SSD drive for example).
>>
>> Thanks for your suggestion,
>> Francois
>>
>>

A buddy of mine created a patch against this tag (the most recent
stable release):

     https://svn.dev.java.net/svn/hudson/tags/hudson-1_382/

The patch allows you to set two additional environment variables
besides HUDSON_HOME:

   HUDSON_WORKSPACES
   HUDSON_BUILDS

Now you can have a Hudson configuration like this:

   HUDSON_HOME=/stable_filesystem/hudson/
   HUDSON_WORKSPACES=/fast_ssd_filesystem/workspaces/
   HUDSON_BUILDS=/slow_sata_high_capacity_raid/builds/


The patch is inline and attached:

Index: core/src/main/java/hudson/model/Hudson.java
===================================================================
--- core/src/main/java/hudson/model/Hudson.java (revision 36300)
+++ core/src/main/java/hudson/model/Hudson.java (working copy)
@@ -30,6 +30,7 @@
 import hudson.BulkChange;
 import hudson.DNSMultiCast;
 import hudson.DescriptorExtensionList;
+import hudson.EnvVars;
 import hudson.Extension;
 import hudson.ExtensionList;
 import hudson.ExtensionListView;
@@ -172,6 +173,9 @@
 import org.xml.sax.InputSource;

 import javax.crypto.SecretKey;
+import javax.naming.Context;
+import javax.naming.InitialContext;
+import javax.naming.NamingException;
 import javax.servlet.RequestDispatcher;
 import javax.servlet.ServletContext;
 import javax.servlet.ServletException;
@@ -1076,6 +1080,8 @@

     private final transient Object updateComputerLock = new Object();

+ private String configuredWorkspaceRoot = null;
+
     /**
      * Updates {@link #computers} by using {@link #getSlaves()}.
      *
@@ -1750,10 +1756,59 @@
         return root;
     }

+//    public FilePath getWorkspaceFor(TopLevelItem item) {
+//        return new FilePath(new File(item.getRootDir(), WORKSPACE_DIRNAME));
+//    }
+
+    private String getConfiguredWorkspaceRoot() {
+     if (configuredWorkspaceRoot == null) {
+            try {
+                InitialContext iniCtxt = new InitialContext();
+                Context env = (Context) iniCtxt.lookup("java:comp/env");
+                String value = (String) env.lookup("HUDSON_WORKSPACES");
+                if(value!=null && value.trim().length()>0) {
+                 configuredWorkspaceRoot = value.trim();
+                 return configuredWorkspaceRoot;
+                }
+                // look at one more place. See issue #1314
+                value = (String) iniCtxt.lookup("HUDSON_WORKSPACES");
+                if(value!=null && value.trim().length()>0) {
+                 configuredWorkspaceRoot = value.trim();
+                 return configuredWorkspaceRoot;
+                }
+            } catch (NamingException e) {
+                // ignore
+            }
+
+            // finally check the system property
+            String sysProp = System.getProperty("HUDSON_WORKSPACES");
+            if(sysProp!=null) {
+             configuredWorkspaceRoot = sysProp.trim();
+             return configuredWorkspaceRoot;
+            }
+
+            // look at the env var next
+            String env = EnvVars.masterEnvVars.get("HUDSON_WORKSPACES");
+            if(env!=null) {
+             configuredWorkspaceRoot = env.trim();
+             return configuredWorkspaceRoot;
+            }
+
+            //not set
+            configuredWorkspaceRoot = "";
+     }
+     return configuredWorkspaceRoot;
+    }
+
     public FilePath getWorkspaceFor(TopLevelItem item) {
-        return new FilePath(new File(item.getRootDir(), WORKSPACE_DIRNAME));
+        if (getConfiguredWorkspaceRoot().equals("")) {
+         return new FilePath(new File(item.getRootDir(), WORKSPACE_DIRNAME));
+        } else {
+         return new FilePath(new File(getConfiguredWorkspaceRoot() +
"/" + item.getName(), WORKSPACE_DIRNAME));
+        }
     }
-
+
+
     public FilePath getRootPath() {
         return new FilePath(getRootDir());
     }
Index: core/src/main/java/hudson/model/Job.java
===================================================================
--- core/src/main/java/hudson/model/Job.java (revision 36300)
+++ core/src/main/java/hudson/model/Job.java (working copy)
@@ -27,6 +27,8 @@
 import static javax.servlet.http.HttpServletResponse.SC_NO_CONTENT;

 import com.infradna.tool.bridge_method_injector.WithBridgeMethods;
+
+import hudson.EnvVars;
 import hudson.ExtensionPoint;
 import hudson.Util;
 import hudson.XmlFile;
@@ -75,6 +77,9 @@
 import java.util.SortedMap;
 import java.util.LinkedList;

+import javax.naming.Context;
+import javax.naming.InitialContext;
+import javax.naming.NamingException;
 import javax.servlet.ServletException;
 import javax.xml.transform.Transformer;
 import javax.xml.transform.TransformerException;
@@ -620,11 +625,44 @@
      * directory for consistency.
      *
      * @see RunMap
+     *
+     * Determines the builds directory for Hudson.
+     *
+     * People makes configuration mistakes, so we are trying to be nice
+     * with those by doing {@link String#trim()}.
      */
     protected File getBuildDir() {
+        // check JNDI for the home directory first
+        try {
+            InitialContext iniCtxt = new InitialContext();
+            Context env = (Context) iniCtxt.lookup("java:comp/env");
+            String value = (String) env.lookup("HUDSON_BUILDS");
+            if(value!=null && value.trim().length()>0)
+                return new File(value.trim());
+            // look at one more place. See issue #1314
+            value = (String) iniCtxt.lookup("HUDSON_BUILDS");
+            if(value!=null && value.trim().length()>0)
+                return new File(value.trim() + "/" + getSearchName());
+        } catch (NamingException e) {
+            // ignore
+        }
+
+        // finally check the system property
+        String sysProp = System.getProperty("HUDSON_BUILDS");
+        if(sysProp!=null)
+            return new File(sysProp.trim() + "/" + getSearchName());
+
+        // look at the env var next
+        String env = EnvVars.masterEnvVars.get("HUDSON_BUILDS");
+        if(env!=null)
+            return new File(env.trim() + "/" +
getSearchName()).getAbsoluteFile();
+
+        // otherwise, use default
         return new File(getRootDir(), "builds");
+
     }
-
+
+
     /**
      * Gets all the runs.
      *

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

hudson-1_382-change_workspace_and_builds-patch.diff (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: 'builds' directory in a separate location?

Jacob Robertson
Without having reviewed the validity of the patch, I vote +1 on the
intent.  This is something we would make use of immediately.  We are
already working around this in a very dumb way by manually setting
each and every job to use a custom workspace.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

AW: 'builds' directory in a separate location?

Harry G
> Without having reviewed the validity of the patch, I vote +1 on the intent.
I also totally agree! Very useful! +1!

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: AW: 'builds' directory in a separate location?

Fred Stluka
+1 from me too.

I wanted daily backups to include the job settings but not all
of the builds, so I tried moving the "builds" folder to a new
location and symlink to it from the standard location. 

However, the logic to maintain the
lastSuccessful and lastStable
links caused a problem.  It created the links at locations:
    jobs/MyJob/builds/../lastSuccessful
   
jobs/MyJob/builds/../lastStable
which is not the same as:
    jobs/MyJob/lastSuccessful
   
jobs/MyJob/lastStable
when "builds" is a symlink.
--Fred
---------------------------------------------------------------------
Fred Stluka -- [hidden email] -- http://bristle.com/~fred/
Bristle Software, Inc -- http://bristle.com -- Glad to be of service!
Open Source: Without walls and fences, we need no Windows or Gates.
---------------------------------------------------------------------


Goettlicher Harald (AA-DGP/ESD2) wrote:
Without having reviewed the validity of the patch, I vote +1 on the intent.
    
I also totally agree! Very useful! +1!

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



  
Reply | Threaded
Open this post in threaded view
|

Re: AW: 'builds' directory in a separate location?

Michael March
So does anyone know the best way to 'propose' this patch to see if it
can get into the trunk?

On Fri, Oct 29, 2010 at 7:16 AM, Fred Stluka <[hidden email]> wrote:

> +1 from me too.
>
> I wanted daily backups to include the job settings but not all
> of the builds, so I tried moving the "builds" folder to a new
> location and symlink to it from the standard location.
>
> However, the logic to maintain the lastSuccessful and lastStable
> links caused a problem.  It created the links at locations:
>     jobs/MyJob/builds/../lastSuccessful
>     jobs/MyJob/builds/../lastStable
> which is not the same as:
>     jobs/MyJob/lastSuccessful
>     jobs/MyJob/lastStable
> when "builds" is a symlink.
>
> --Fred
> ---------------------------------------------------------------------
> Fred Stluka -- mailto:[hidden email] -- http://bristle.com/~fred/
> Bristle Software, Inc -- http://bristle.com -- Glad to be of service!
> Open Source: Without walls and fences, we need no Windows or Gates.
> ---------------------------------------------------------------------
>
>
> Goettlicher Harald (AA-DGP/ESD2) wrote:
>
> Without having reviewed the validity of the patch, I vote +1 on the intent.
>
>
> I also totally agree! Very useful! +1!

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]